5. Comparing Independent Samples with Unequal Variances

If the assumption of equal variances is violated, you have to compute the approximate t using individual sample variances instead of the pooled variance. It is also necessary to obtain the Satterthwaite’s, Cochran-Cox’s (SAS), or Welch’s (Stata) approximation of the degrees of freedom. However, the null hypothesis remains unchanged. This section compares mean death rates from kidney cancer between the west/south areas and the east/north counterparts.

5.1 Independent Sample T-test in STATA

Let us check equality of variances using the .oneway command. The tabulate option produces a table of summary statistics of the groups. The west and south areas (west=1) have a lower average death rate of 2.6183 but a larger standard deviation of .5984 than the east and north counterparts.

The chi-squared 8.6506 of the Bartlett’s test rejects the null hypothesis of equal variances (p<.003). The folded F statistic is 3.9749=.59837219^2/.3001298^2 with 23 and 19 degrees of freedom, which also rejects the null hypothesis (p<.0034). Therefore, the approximate t is computed using individual variances.

The approximate t of 2.7819 rejects the null hypothesis of no difference in mean death rates between the two areas (p<.0086). The c is .2318 and Satterthwaite’s approximation of the degrees of freedom is 35.1071.

In Stata, you have to use the unequal option in the .ttest command to perform the t-test for two independent samples with unequal variances.

The approximate t 2.7817 and Satterthwaite’s approximation 35.1098, due to the rounding error, are slight different from what is manually computed above. Notice that the 2.7817 is the square root of the F statistic 6.92 of the .oneway output above. If you want to get Welch’s approximation, use the welch as well as unequal options; without the unequal option, welch is ignored.

Satterthwaite’s approximation is slightly smaller than Welch’s 36.2258. Again, these approximations are not integers, but real numbers. The approximate t 2.7817 remains unchanged, but the p-value becomes slightly smaller due to the different approximation used. However, both tests reject the null hypothesis of equal population means at the .05 level. The north and east areas have larger average death rates from kidney cancer than the south and west.

For aggregated data, again use the .ttesti command with the necessary options.

As mentioned earlier, the unpaired option of the .ttest command directly compares two variables without data manipulation. The option treats the two variables arranged in the second type of data arrangement in Figure 3 as being independent of each other. The following example compares the average death rates from bladder and kidney cancers using the both unpaired and unequal options.

The death rate from bladder cancer has larger mean (4.1211) and standard deviation (.9649) than that from kidney cancer. Their variances do not appear equal. The F 3.4556 = (.9649249^2) / (.5190799^2) rejects the null hypothesis of equal variances (p<.0001). Accordingly, the approximate t is computed with the degrees of freedom are approximated as well. If the welch option is omitted in the .ttest above, Satterthwaite's degrees of freedom 65.9643 will be produced instead.

For aggregated data, again, use the .ttesti command without the unpaired option.

. ttesti 44 4.121136 .9649249 44 2.794545 .5190799, unequal welch level(95)

The approximate t 8.0312 is very large and rejects the null hypothesis of equal average death rates (p<.0000). The death rate from bladder cancer appears to have a larger population mean than the rate from kidney cancer.

Top

5.2 Independent Sample T-test in SAS

The TTEST procedure reports statistics for both cases of equal and unequal variances. The COCHRAN option adds the Cochran-Cox’s approximation of the degrees of freedom to the output.

The F 3.97 rejects the null hypothesis of equal variances (p<.0034). Thus, individual sample variances are used to compute the approximate t and the approximation of the degrees of freedom needs to be used in statistical inferences. In the above output, Cochran-Cox’s approximation is not defined since sample sizes are not equal. The t’ 2.78 is large enough to reject the null hypothesis of the same population death rate (p<.0086). If you mistakenly read the smaller t statistic 2.63 for the pooled variance (p<.0118), you might not reject the null hypothesis at the .01 level.

Now, let us compare the average death rates from bladder and kidney cancers that are arranged in the second type of Figure 3. We need to reshape the data set first to make it into the first data arrangement type.

The CLASS statement specifies a categorical variable (grouping variable) bla_kid and the new variable rate contains both death rates from bladder and kidney cancers. Unlike the Stata .ttest command, the SAS TTEST procedure allows you to specify a hypothesized value other than zero.

The 3.46 of the F test rejects the null hypothesis of equal variances (p<.0001). The t statistic 8.03 for the case of unequal variances is large and rejects the null hypothesis of the same population mean at the .01 level. The population means of the two death rates are different. Interestingly, these t statistics in this case happen to be same regardless whether the variances are equal or not. However, the degrees of freedom vary across approximation methods. Remember the Welch’s approximation in Stata is 67.0324.

Top

5.3 Independent Sample T-test in SPSS

Like SAS, SPSS reports t statistics for both cases of equal and unequal variances. Levene's F 5.466 rejects the null hypothesis of equal variances at the .05 level (p<.024). Other statistics are the same as those of SAS.

T-TEST
   /GROUPS = west(0 1)
   /VARIABLES = kidney
   /CRITERIA = CI(.95) .

Table 4 summarizes the independent sample t-test in Stata, SAS, and SPSS. These software packages respectively report the Bartlett’s test, the folded form F test, and Levene’s F test. Despite different test statistics in Stata and SAS, their p-values are almost same. Stata and SAS additionally produce Welch’s and Cochran-Cox’s approximations of the degrees of freedom, respectively. The significant difference in population variances produces different t and t’ and requires approximations of the degrees of freedom.

While the t’ 2.7817 rejects the null hypothesis at the .01 level, the t statistic of 2.6311 do not. Heavy cigarette consuming states have a higher average death rate from kidney cancer than the light counterparts. If you mistakenly read the latter, you may reach a misleading conclusion at the .01 level.

Table 4. Summary of the T-test for Independent Samples with Unequal Variances (Kidney Cancer)