Stat/Math
Software Support
Software Consulting
Software Availability
Software Price
Contact

User Support
Documentation
Knowledge Base
Education
Consulting
Podcasts

Systems & Services
Cyberinfrastructure
Supercomputers
Grid Computing
Storage
Visualization
Digital Libraries & Data

Results & Impact
Publications
Grants & Grant Info
Events & Outreach
Economic Impact
Survey Results

Vision & Planning
News & Features

# 5. Comparing Independent Samples with Unequal Variances

If the assumption of equal variances is violated, you have to compute the approximate t using individual sample variances instead of the pooled variance. It is also necessary to obtain the Satterthwaite’s, Cochran-Cox’s (SAS), or Welch’s (Stata) approximation of the degrees of freedom. However, the null hypothesis remains unchanged. This section compares mean death rates from kidney cancer between the west/south areas and the east/north counterparts.

5.1 Independent Sample T-test in STATA

Let us check equality of variances using the .oneway command. The tabulate option produces a table of summary statistics of the groups. The west and south areas (west=1) have a lower average death rate of 2.6183 but a larger standard deviation of .5984 than the east and north counterparts.

. oneway kidney west, tabulate
|          Summary of kidney
west |        Mean   Std. Dev.       Freq.
------------+------------------------------------
0 |       3.006    .3001298          20
1 |   2.6183333   .59837219          24
------------+------------------------------------
Total |   2.7945455   .51907993          44

Analysis of Variance
Source              SS         df      MS            F     Prob > F
------------------------------------------------------------------------
Between groups      1.63947758      1   1.63947758      6.92     0.0118
Within groups      9.94661333     42   .236824127
------------------------------------------------------------------------
Total           11.5860909     43   .269443975

Bartlett's test for equal variances:  chi2(1) =   8.6506  Prob>chi2 = 0.003

The chi-squared 8.6506 of the Bartlett’s test rejects the null hypothesis of equal variances (p<.003). The folded F statistic is 3.9749=.59837219^2/.3001298^2 with 23 and 19 degrees of freedom, which also rejects the null hypothesis (p<.0034). Therefore, the approximate t is computed using individual variances.

The approximate t of 2.7819 rejects the null hypothesis of no difference in mean death rates between the two areas (p<.0086). The c is .2318 and Satterthwaite’s approximation of the degrees of freedom is 35.1071.

In Stata, you have to use the unequal option in the .ttest command to perform the t-test for two independent samples with unequal variances.

. ttest kidney, by(west) unequal level(95)
Two-sample t test with unequal variances
------------------------------------------------------------------------------
Group |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
0 |      20       3.006    .0671111    .3001298    2.865535    3.146465
1 |      24    2.618333    .1221422    .5983722    2.365663    2.871004
---------+--------------------------------------------------------------------
combined |      44    2.794545    .0782542    .5190799    2.636731     2.95236
---------+--------------------------------------------------------------------
diff |            .3876667     .139365                .1047722    .6705611
------------------------------------------------------------------------------
diff = mean(0) - mean(1)                                      t =   2.7817
Ho: diff = 0                     Satterthwaite's degrees of freedom =  35.1098

Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
Pr(T < t) = 0.9957         Pr(|T| > |t|) = 0.0086          Pr(T > t) = 0.0043

The approximate t 2.7817 and Satterthwaite’s approximation 35.1098, due to the rounding error, are slight different from what is manually computed above. Notice that the 2.7817 is the square root of the F statistic 6.92 of the .oneway output above. If you want to get Welch’s approximation, use the welch as well as unequal options; without the unequal option, welch is ignored.

. ttest kidney, by(west) unequal welch
Two-sample t test with unequal variances
------------------------------------------------------------------------------
Group |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
0 |      20       3.006    .0671111    .3001298    2.865535    3.146465
1 |      24    2.618333    .1221422    .5983722    2.365663    2.871004
---------+--------------------------------------------------------------------
combined |      44    2.794545    .0782542    .5190799    2.636731     2.95236
---------+--------------------------------------------------------------------
diff |            .3876667     .139365                .1050824    .6702509
------------------------------------------------------------------------------
diff = mean(0) - mean(1)                                      t =   2.7817
Ho: diff = 0                             Welch's degrees of freedom =  36.2258

Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
Pr(T < t) = 0.9957         Pr(|T| > |t|) = 0.0085          Pr(T > t) = 0.0043

Satterthwaite’s approximation is slightly smaller than Welch’s 36.2258. Again, these approximations are not integers, but real numbers. The approximate t 2.7817 remains unchanged, but the p-value becomes slightly smaller due to the different approximation used. However, both tests reject the null hypothesis of equal population means at the .05 level. The north and east areas have larger average death rates from kidney cancer than the south and west.

For aggregated data, again use the .ttesti command with the necessary options.

. ttesti 20 3.006 .3001298 24 2.618333 .5983722, unequal welch

As mentioned earlier, the unpaired option of the .ttest command directly compares two variables without data manipulation. The option treats the two variables arranged in the second type of data arrangement in Figure 3 as being independent of each other. The following example compares the average death rates from bladder and kidney cancers using the both unpaired and unequal options.

. ttest bladder=kidney, unpaired unequal welch
Two-sample t test with unequal variances
------------------------------------------------------------------------------
Variable |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
bladder |      44    4.121136    .1454679    .9649249    3.827772      4.4145
kidney |      44    2.794545    .0782542    .5190799    2.636731     2.95236
---------+--------------------------------------------------------------------
combined |      88    3.457841    .1086268    1.019009    3.241933    3.673748
---------+--------------------------------------------------------------------
diff |            1.326591    .1651806                .9968919     1.65629
------------------------------------------------------------------------------
diff = mean(bladder) - mean(kidney)                           t =   8.0312
Ho: diff = 0                             Welch's degrees of freedom =  67.0324

Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
Pr(T < t) = 1.0000         Pr(|T| > |t|) = 0.0000          Pr(T > t) = 0.0000

The death rate from bladder cancer has larger mean (4.1211) and standard deviation (.9649) than that from kidney cancer. Their variances do not appear equal. The F 3.4556 = (.9649249^2) / (.5190799^2) rejects the null hypothesis of equal variances (p<.0001). Accordingly, the approximate t is computed with the degrees of freedom are approximated as well. If the welch option is omitted in the .ttest above, Satterthwaite's degrees of freedom 65.9643 will be produced instead.

For aggregated data, again, use the .ttesti command without the unpaired option.

. ttesti 44 4.121136 .9649249 44 2.794545 .5190799, unequal welch level(95)

The approximate t 8.0312 is very large and rejects the null hypothesis of equal average death rates (p<.0000). The death rate from bladder cancer appears to have a larger population mean than the rate from kidney cancer.

Top

5.2 Independent Sample T-test in SAS

The TTEST procedure reports statistics for both cases of equal and unequal variances. The COCHRAN option adds the Cochran-Cox’s approximation of the degrees of freedom to the output.

PROC TTEST COCHRAN DATA=masil.smoking;
CLASS west;
VAR kidney;
RUN;
The TTEST Procedure

Statistics

Lower CL          Upper CL  Lower CL           Upper CL
Variable  s_west            N      Mean    Mean      Mean   Std Dev  Std Dev   Std Dev

kidney               0     20    2.8655   3.006    3.1465    0.2282   0.3001    0.4384
kidney               1     24    2.3657  2.6183     2.871    0.4651   0.5984    0.8394
kidney    Diff (1-2)             0.0903  0.3877     0.685    0.4013   0.4866    0.6185

Statistics

Variable    west        Std Err    Minimum    Maximum

kidney               0   0.0671       2.34       3.62
kidney               1   0.1221       1.59       4.32
kidney    Diff (1-2)     0.1473

T-Tests

Variable    Method           Variances      DF    t Value    Pr > |t|

kidney      Pooled           Equal          42       2.63      0.0118
kidney      Satterthwaite    Unequal      35.1       2.78      0.0086
kidney      Cochran          Unequal         .       2.78      0.0109

Equality of Variances

Variable    Method      Num DF    Den DF    F Value    Pr > F

kidney      Folded F        23        19       3.97    0.0034

The F 3.97 rejects the null hypothesis of equal variances (p<.0034). Thus, individual sample variances are used to compute the approximate t and the approximation of the degrees of freedom needs to be used in statistical inferences. In the above output, Cochran-Cox’s approximation is not defined since sample sizes are not equal. The t’ 2.78 is large enough to reject the null hypothesis of the same population death rate (p<.0086). If you mistakenly read the smaller t statistic 2.63 for the pooled variance (p<.0118), you might not reject the null hypothesis at the .01 level.

Now, let us compare the average death rates from bladder and kidney cancers that are arranged in the second type of Figure 3. We need to reshape the data set first to make it into the first data arrangement type.

DATA masil.smoking3;
SET masil.smoking;
rate = kidney; bla_kid ='Kidney'; OUTPUT;
KEEP bla_kid rate;
RUN;

PROC TTEST COCHRAN DATA=masil.smoking3;
CLASS bla_kid;
VAR rate;
RUN;

The CLASS statement specifies a categorical variable (grouping variable) bla_kid and the new variable rate contains both death rates from bladder and kidney cancers. Unlike the Stata .ttest command, the SAS TTEST procedure allows you to specify a hypothesized value other than zero.

The TTEST Procedure

Statistics

Lower CL          Upper CL  Lower CL           Upper CL
Variable  bla_kid         N      Mean    Mean      Mean   Std Dev  Std Dev   Std Dev  Std Err

rate      Bladder        44    3.8278  4.1211    4.4145    0.7972   0.9649    1.2226   0.1455
rate      Kidney         44    2.6367  2.7945    2.9524    0.4289   0.5191    0.6577   0.0783
rate      Diff (1-2)           0.9982  1.3266     1.655    0.6743   0.7748    0.9107   0.1652

T-Tests

Variable    Method           Variances      DF    t Value    Pr > |t|

rate        Pooled           Equal          86       8.03      <.0001
rate        Satterthwaite    Unequal        66       8.03      <.0001
rate        Cochran          Unequal        43       8.03      <.0001

Equality of Variances

Variable    Method      Num DF    Den DF    F Value    Pr > F

rate        Folded F        43        43       3.46    <.0001

The 3.46 of the F test rejects the null hypothesis of equal variances (p<.0001). The t statistic 8.03 for the case of unequal variances is large and rejects the null hypothesis of the same population mean at the .01 level. The population means of the two death rates are different. Interestingly, these t statistics in this case happen to be same regardless whether the variances are equal or not. However, the degrees of freedom vary across approximation methods. Remember the Welch’s approximation in Stata is 67.0324.

Top

5.3 Independent Sample T-test in SPSS

Like SAS, SPSS reports t statistics for both cases of equal and unequal variances. Levene's F 5.466 rejects the null hypothesis of equal variances at the .05 level (p<.024). Other statistics are the same as those of SAS.

T-TEST
/GROUPS = west(0 1)
/VARIABLES = kidney
/CRITERIA = CI(.95) .

Table 4 summarizes the independent sample t-test in Stata, SAS, and SPSS. These software packages respectively report the Bartlett’s test, the folded form F test, and Levene’s F test. Despite different test statistics in Stata and SAS, their p-values are almost same. Stata and SAS additionally produce Welch’s and Cochran-Cox’s approximations of the degrees of freedom, respectively. The significant difference in population variances produces different t and t’ and requires approximations of the degrees of freedom.

While the t’ 2.7817 rejects the null hypothesis at the .01 level, the t statistic of 2.6311 do not. Heavy cigarette consuming states have a higher average death rate from kidney cancer than the light counterparts. If you mistakenly read the latter, you may reach a misleading conclusion at the .01 level.

Table 4. Summary of the T-test for Independent Samples with Unequal Variances (Kidney Cancer)