Comparing Group Means: The T-test and One-way ANOVA Using STATA, SAS, and SPSS
Hun Myoung Park
Software Consultant, UITS Center for Statistical and Mathematical Computing
Contact: kucc625 at indiana.edu
Last Updated on August 15, 2005
Table of Contents
(See the PDF format)
- Introduction
- Univariate Samples
- Paired (Dependent) Samples
- Independent Samples with Equal Variances
- Independent Samples with Unequal Variances
- One-way ANOVA, GLM, and Regression
- Conclusion
- Appendix
- References
This document summarizes the method of comparing group means and illustrates how to conduct the t-test and one-way ANOVA using STATA 9.0, SAS 9.1, and SPSS 13.0.
The t-test and analysis of variance (ANOVA) compare group means. The mean of a variable to be compared should be substantively interpretable. A t-test may examine gender differences in average salary or racial (white versus black) differences in average annual income. The left-hand side (LHS) variable to be tested should be interval or ratio, whereas the right-hand side (RHS) variable should be binary (categorical).
While the t-test is limited to comparing means of two groups, one-way ANOVA can compare more than two groups. Therefore, the t-test is considered a special case of one-way ANOVA. These analyses do not, however, necessarily imply any causality (i.e., a causal relationship between the left-hand and right-hand side variables). Table 1 compares the t-test and one-way ANOVA.
Table 1. Comparison between the T-test and One-way ANOVA
|
T-test |
One-way ANOVA |
| LHS (Dependent) |
Interval or ratio variable |
Interval or ratio variable |
| RHS (Independent) |
Binary variable with only two groups |
Categorical variable |
| Null Hypothesis |
mu1=mu2 |
mu1=mu2=mu3 ... |
| Probability Distribution* |
T Distribution |
F distribution |
* In the case of one degree of freedom on numerator, F=t^2.
The t-test assumes that samples are randomly drawn from normally distributed populations with unknown population means. Otherwise, the means are no longer the best measures of central tendency and the t-test will not be valid. The Central Limit Theorem says, however, that the distributions of sample means are approximately normal when N is large. When n1 + n2 >=30, in practice, you do not need to worry too much about the normality assumption.
You may numerically test the normality assumption using the Shapiro-Wilk W (N<=2000), Shapiro-Francia W (N<=5000), Kolmogorov-Smirnov D (N>2000), and Jarque-Bera tests. If N is small and the null hypothesis of normality is rejected, you my try such nonparametric methods as the Kolmogorov-Smirnov test, Kruscal-Wallis test, Wilcoxon Rank-Sum Test, or Log-Rank Test, depending on the circumstances.
In STATA, the .ttest and .ttesti commands are used to conduct t-tests, whereas the .anova and .oneway commands perform one-way ANOVA. SAS has the TTEST procedure for t-test, but the UNIVARIATE, and MEANS procedures also have options for t-test. SAS provides various procedures for the analysis of variance, such as the ANOVA, GLM, and MIXED procedures. The ANOVA procedure can handle balanced data only, while the GLM and MIXED can analyze either balanced or unbalanced data (having the same or different numbers of observations across groups). However, unbalanced data does not cause any problems in the t-test and one-way ANOVA. In SPSS, T-TEST, ONEWAY, and UNIANOVA commands are used to perform t-test and one-way ANOVA.
Table 2 summarizes STATA commands, SAS procedures, and SPSS commands that are associated with t-test and one-way ANOVA.
Table 2. Related Procedures and Commands in STATA, SAS, and SPSS
| |
STATA 9.0 SE |
SAS 9.1 |
SPSS 13.0 |
| Normality Test |
.sktest; .swilk;. sfrancia |
UNIVARIATE |
EXAMINE |
| Equal Variance |
.oneway |
TTEST |
T-TEST |
| Nonparametric |
.ksmirnov; .kwallis |
NPAR1WAY |
NPARTESTS |
| T-test |
.ttest |
TTEST; MEANS; ANOVA |
T-TEST |
| ANOVA |
.anova; .oneway |
ANOVA |
ONEWAY |
| GLM* |
|
GLM; MIXED |
UNIANOVA |
* The STATA .glm command is not used for the T test, but for the generalized linear model.
There are two types of data arrangement for t-tests (Figure 1). The first data arrangement has a variable to be tested and a grouping variable to classify groups (0 or 1). The second, appropriate especially for paired samples, has two variables to be tested. The two variables in this type are not, however, necessarily paired nor balanced. SAS and SPSS prefer the first data arrangement, whereas STATA can handle either type flexibly. Note that the numbers of observations across groups are not necessarily equal.
Figure 1. Two Types of Data Arrangement
| Variable |
Group |
|
|
Variable1 |
Variable2 |
| x |
0 |
| |
x |
y |
| x |
0 |
| |
x |
y |
| ... |
... |
| |
... |
... |
| y |
1 |
| | | |
| y |
1 |
| | | |
| ... |
... |
| | | |
The data set used here is adopted from J. F. Fraumeni's study on cigarette smoking and cancer (Fraumeni 1968). The data are per capita numbers of cigarettes sold by 43 states and the District of Columbia in 1960 together with death rates per hundred thousand people from various forms of cancer. Two variables were added to categorize states into two groups. See the appendix for the details.
Top
The univariate-sample or one-sample t-test determines whether an unknown population mean mu differs from a hypothesized value c that is commonly set to zero. The t statistic follows Student's T probability distribution with n-1 degrees of freedom,
, where y is a variable to be tested and n is the number of observations.
Suppose you want to test if the population mean of the death rates from lung cancer is 20 per 100,000 people at the .01 significance level. Note the default significance level used in most software is the .05 level.
2.1 T-test in STATA
The .ttest command conducts T-tests in an easy and flexible manner. For a univariate sample test, the command requires that a hypothesized value be explicitly specified. The level() option indicates the confidence level as a percentage. The 99 percent confidence level is equivalent to the .01 significance level.
. ttest lung=20, level(99)
One-sample t test
------------------------------------------------------------------------------
Variable
| Obs Mean
Std. Err. Std. Dev. [99% Conf. Interval]
---------+--------------------------------------------------------------------
lung | 44 19.65318
.6374133 4.228122 17.93529
21.37108
------------------------------------------------------------------------------
mean = mean(lung)
t
= -0.5441
Ho: mean = 20
degrees of freedom = 43
Ha: mean < 20
Ha: mean != 20
Ha: mean > 20
Pr(T < t) = 0.2946 Pr(|T| >
|t|) = 0.5892 Pr(T >
t) = 0.7054
STATA first lists descriptive statistics of the variable lung. The mean and standard deviation of the 44 observations are 19.653 and 4.228, respectively. The T statistic is -.544 = (19.653-20) / .6374. Finally, the degrees of freedom are 43 =44-1.
There are three T-tests at the bottom of the output above. The first and third are one-tailed tests, whereas the second is a two-tailed test. The t statistic -.544 and its large p-value do not reject the null hypothesis that the population mean of the death rate from lung cancer is 20 at the .01 level. The mean of the death rate may be 20 per 100,000 people. Note that the hypothesized value 20 falls into the 99 percent confidence interval 17.935-21.371.
If you just have the aggregate data (i.e., the number of observations, mean, and standard deviation of the sample), use the .ttesti command to replicate the t-test above. Note the hypothesized value is specified at the end of the summary statistics.
. ttesti 44 19.65318 4.228122 20, level(99)
2.2 T-test Using the SAS TTEST Procedure
The TTEST procedure conducts various types of t-tests in SAS. The H0 option specifies a hypothesized value, whereas the ALPHA indicates a significance level. If omitted, the default values zero and .05 respectively are assumed.
PROC TTEST H0=20 ALPHA=.01 DATA=masil.smoking;
VAR lung;
RUN;
The TTEST Procedure
Statistics
Lower CL Upper CL
Lower CL Upper CL
Variable N Mean
Mean Mean Std Dev Std Dev
Std Dev Std Err
lung 44 17.935
19.653 21.371 3.2994 4.2281
5.7989 0.6374
T-Tests
Variable
DF t Value Pr > |t|
lung 43
-0.54 0.5892
The TTEST procedure reports descriptive statistics followed by a one-tailed t-test. You may have a summary data set containing the values of a variable (lung) and their frequencies (count). The FREQ option of the TTEST procedure provides the solution for this case.
PROC TTEST H0=20 ALPHA=.01 DATA=masil.smoking;
VAR lung;
FREQ count;
RUN;
2.3 T-test Using the SAS UNIVARIATE and MEANS Procedures
The SAS UNIVARIATE and MEANS procedures also conduct a t-test for a univariate-sample. The UNIVARIATE procedure is basically designed to produces a variety of descriptive statistics of a variable. Its MU0 option tells the procedure to perform a t-test using the hypothesized value specified. The VARDEF=DF specifies a divisor (degrees of freedom) used in computing the variance (standard deviation). The NORMAL option examines if the variable is normally distributed.
PROC UNIVARIATE MU0=20 VARDEF=DF NORMAL ALPHA=.01 DATA=masil.smoking;
VAR lung;
RUN;
The UNIVARIATE Procedure
Variable: lung
Moments
N
44 Sum Weights
44
Mean
19.6531818 Sum Observations
864.74
Std Deviation 4.22812167 Variance
17.8770129
Skewness -0.104796
Kurtosis
-0.949602
Uncorrected SS 17763.604 Corrected
SS 768.711555
Coeff Variation 21.5136751 Std Error Mean
0.63741333
Basic Statistical Measures
Location
Variability
Mean 19.65318 Std Deviation
4.22812
Median 20.32000 Variance
17.87701
Mode .
Range
15.26000
Interquartile Range 6.53000
Tests for Location: Mu0=20
Test -Statistic-
-----p Value------
Student's
t t -0.5441 Pr > |t|
0.5892
Sign M
1 Pr >= |M| 0.8804
Signed Rank S -36.5
Pr >= |S| 0.6752
Tests for Normality
Test
--Statistic--- -----p Value------
Shapiro-Wilk W
0.967845 Pr < W 0.2535
Kolmogorov-Smirnov D 0.086184
Pr > D >0.1500
Cramer-von
Mises W-Sq 0.063737 Pr
> W-Sq >0.2500
Anderson-Darling A-Sq 0.382105
Pr > A-Sq >0.2500
Quantiles (Definition 5)
Quantile Estimate
100% Max 27.270
99%
27.270
95%
25.950
90%
25.450
75% Q3 22.815
50% Median 20.320
25% Q1 16.285
Quantiles (Definition 5)
Quantile
Estimate
10%
14.110
5%
12.120
1%
12.010
0% Min
12.010
Extreme Observations
-----Lowest---- ----Highest----
Value Obs
Value Obs
12.01
39 25.45
16
12.11 33
25.88 1
12.12 30
25.95 27
13.58 10
26.48 18
14.11
36 27.27
8
The third block of the output above reports a t statistic and its p-value. The fourth block contains several statistics of normality test. Since N is less than 2,000, you should read the Shapiro-Wilk W, which suggests that lung is normally distributed (p<.2535)
The MEANS procedure also conducts t-tests using the T and PROBT options that request the t statistic and its two-tailed p-value. The CLM option produces the two-tailed confidence interval (or upper and lower limits). The MEAN, STD, and STDERR respectively print the sample mean, standard deviation, and standard error.
PROC MEANS MEAN STD STDERR T PROBT CLM VARDEF=DF ALPHA=.01 DATA=masil.smoking;
VAR lung;
RUN;
The MEANS Procedure
Analysis Variable : lung
Lower 99% Upper 99%
Mean Std Dev Std
Error t Value Pr > |t| CL for Mean CL
for Mean
---------------------------------------------------------------------------------------
19.6531818 4.2281217 0.6374133
30.83 <.0001 17.9352878
21.3710758
---------------------------------------------------------------------------------------
The MEANS procedure does not, however, have an option to specify a hypothesized value to anything other than zero. Thus, the null hypothesis here is that the population mean of death rate from lung cancer is zero. The t statistic 30.83 is (19.6532-0)/.6374. The large t statistic and small p-value reject the null hypothesis, reporting a consistent conclusion.
2.4 T-test in SPSS
The SPSS has the T-TEST command for t-tests. The /TESTVAL subcommand specifies the value with which the sample mean is compared, whereas the /VARIABLES list the variables to be tested. Like STATA, SPSS specifies a confidence level rather than a significance level in the /CRITERIA=CI() subcommand.
T-TEST
/TESTVAL = 20
/VARIABLES = lung
/MISSING = ANALYSIS
/CRITERIA = CI(.99) .
Top
When two variables are not independent, but paired, the difference of these two variables is treated as if it were a single sample. This test is appropriate for pre-post treatment responses. The null hypothesis is that the true mean difference of the two variables is D0. The difference is typically assumed to be zero unless explicitly specified.
3.1 T-test in STATA
In order to conduct a paired sample t-test, you need to list two variables separated by an equal sign. The interpretation of the t-test remains almost unchanged. The -1.871 = (-10.1667-0)/5.4337 at 35 degrees of freedom does not reject the null hypothesis that the difference is zero.
. ttest pre=post0, level(95)
Paired t test
------------------------------------------------------------------------------
Variable
| Obs Mean
Std. Err. Std. Dev. [95% Conf. Interval]
---------+--------------------------------------------------------------------
pre | 36 176.0278
6.529723 39.17834 162.7717
189.2838
post0 | 36 186.1944
7.826777 46.96066 170.3052
202.0836
---------+--------------------------------------------------------------------
diff | 36 -10.16667
5.433655 32.60193 -21.19757
.8642387
------------------------------------------------------------------------------
mean(diff) = mean(pre ? post0) t
= -1.8711
Ho:
mean(diff) = 0
degrees of freedom = 35
Ha:
mean(diff) < 0
Ha: mean(diff) != 0
Ha: mean(diff) > 0
Pr(T
< t) = 0.0349 Pr(|T| >
|t|) = 0.0697 Pr(T >
t) = 0.9651
Alternatively, you may first compute the difference between the two variables, and then conduct one-sample t-test. Note that the default confidence level, level(95), can be omitted.
. gen d=pre-post0
. ttest d=0
3.2 T-test in SAS
In the TTEST procedure, you have to use the PAIRED instead of the VAR statement. For the output of the following procedure, refer to the end of this section.
PROC TTEST DATA=temp.drug;
PAIRED pre*post0;
RUN;
The PAIRED statement provides various ways of comparing variables using asterisk (*) and colon (:) operators. The asterisk requests comparisons between each variable on the left with each variable on the right. The colon requests comparisons between the first variable on the left and the first on the right, the second on the left and the second on the right, and so forth.
Consider the following examples.
PROC TTEST;
PAIRED pro: post0;
PAIRED (a b)*(c d); /* Equivalent to PAIRED a*c a*d b*c b*d; */
PAIRED (a b):(c d); /* Equivalent to PAIRED a*c b*c; */
PAIRED (a1-a10)*(b1-b10);
RUN;
The first PAIRED statement is the same as the PAIRED pre*post0. The second and the third PAIRED statements contrast differences between asterisk and colon operators. The hyphen (-) operator in the last statement indicates a1 through a10 and b1 through b10. Let us consider an example of the PAIRED statement.
PROC TTEST DATA=temp.drug;
PAIRED (pre)*(post0-post1)
RUN;
The TTEST Procedure
Statistics
Lower CL Upper CL
Lower CL Upper CL
Difference
N Mean Mean
Mean Std Dev Std Dev Std Dev Std Err
pre
- post0 36 -21.2
-10.17 0.8642 26.443 32.602
42.527 5.4337
pre
- post1 36 -30.43
-20.39 -10.34 24.077 29.685
38.723 4.9475
T-Tests
Difference DF
t Value Pr > |t|
pre - post0 35
-1.87 0.0697
pre - post1 35
-4.12 0.0002
The first t statistic for pre versus post0 is identical to that of the previous section. The second for pre versus post1 rejects the null hypothesis of no mean difference at the .01 level (p<.0002).
In order to use the UNIVARIATE and MEANS procedures, the difference between two paired variables should be computed in advance.
DATA temp.drug2;
SET temp.drug;
d1 = pre - post0;
d2 = pre - post1;
RUN;
PROC UNIVARIATE MU0=0 VARDEF=DF NORMAL; VAR d1 d2; RUN;
PROC MEANS MEAN STD STDERR T PROBT CLM; VAR d1 d2; RUN;
PROC TTEST ALPHA=.05; VAR d1 d2; RUN;
3.3 T-test in SPSS
In SPSS, the PAIRS subcommand indicates a paired sample t-test.
T-TEST PAIRS = pre post0
/CRITERIA = CI(.95)
/MISSING = ANALYSIS .
Top
You should check three assumptions first when testing the mean difference of two independent samples. First, the samples are drawn from normally distributed populations with unknown parameters. Second, the two samples are independent in the sense that they are drawn from different populations and/or the elements of one sample are not related to those of the other sample. Finally, the population variances of the two groups, and are equal. If any one of assumption is violated, the t-test is not valid.
An example here is to compare mean death rates from lung cancer between smokers and non-smokers. Let us begin with discussing the equal variance assumption.
4.1 F test for Equal Variances
The folded form F test is widely used to examine whether two populations have the same variance. The statistic is
, where L and S respectively indicate groups with larger and smaller sample variances. Unless the null hypothesis of equal variances is rejected, the pooled variance estimate is used. The null hypothesis of the independent sample t-test is mu1-mu2=D0.
When the assumption is violated, the t-test requires the approximations of the degree of freedom. The null hypothesis and other components of the t-test, however, remain unchanged. Satterthwaite's approximation for the degree of freedom is commonly used. Note that the approximation is a real number, not an integer.
The SAS TTEST procedure and SPSS T-TEST command conduct F tests for equal variance. SAS reports the folded form F statistic, whereas SPSS computes Levene's weighted F statistic. In STATA, the .oneway command produces Bartlett's statistic for the equal variance test. The following is an example of Bartlett's test that does not reject the null hypothesis of equal variance.
. oneway lung smoke
Analysis of Variance
Source
SS df
MS F
Prob > F
------------------------------------------------------------------------
Between
groups 313.031127
1 313.031127 28.85
0.0000
Within groups 455.680427 42
10.849534
------------------------------------------------------------------------
Total 768.711555
43 17.8770129
Bartlett's test for equal
variances: chi2(1) = 0.1216 Prob>chi2 = 0.727
STATA, SAS, and SPSS all compute Satterthwaite's approximation of the degrees of freedom. In addition, the SAS TTEST procedure reports Cochran-Cox approximation and the STATA .ttest command provides Welch's degrees of freedom.
4.2 T-test in STATA
With the .ttest command, you have to specify a grouping variable smoke in this example in the parenthesis of the by option.
. ttest lung, by(smoke) level(95)
Two-sample t test with equal variances
------------------------------------------------------------------------------
Group | Obs
Mean Std. Err. Std. Dev. [95% Conf.
Interval]
---------+--------------------------------------------------------------------
0 | 22 16.98591
.6747158 3.164698 15.58276
18.38906
1 | 22 22.32045
.7287523 3.418151 20.80493
23.83598
---------+--------------------------------------------------------------------
combined
| 44 19.65318
.6374133 4.228122 18.36772
20.93865
---------+--------------------------------------------------------------------
diff | -5.334545
.9931371
-7.338777 -3.330314
------------------------------------------------------------------------------
diff = mean(0) - mean(1)
t = -5.3714
Ho:
diff = 0
degrees of freedom = 42
Ha: diff < 0 Ha:
diff != 0
Ha: diff > 0
Pr(T
< t) = 0.0000 Pr(|T| >
|t|) = 0.0000 Pr(T >
t) = 1.0000
Let us first check the equal variance. The F statistic is 1.17=3.4182^2/3.1647^2~F(21,21). The degrees of freedom of the numerator and denominator are 21 (=22-1). The p-value of .7273, virtually the same as that of Bartlett's test above, does not reject the null hypothesis of equal variance. Thus, the t-test here is valid (t=-5.3714 and p<.0000).
If only aggregate data of the two variables are available, use the .ttesti command and list the number of observations, mean, and standard deviation of the two variables.
. ttesti 22 16.85591 3.164698 22 22.32045 3.418151, level(95)
Suppose a data set is differently arranged (second type in Figure 1) so that one variable smk_lung has data for smokers and the other non_lung for non-smokers. You have to use the unpaired option to indicate that two variables are not paired. A grouping variable here is not necessary. Compare the following output with what is printed above.
. ttest smk_lung=non_lung, unpaired
Two-sample t test with equal variances
------------------------------------------------------------------------------
Variable
| Obs Mean
Std. Err. Std. Dev. [95% Conf. Interval]
---------+--------------------------------------------------------------------
smk_lung
| 22 22.32045
.7287523 3.418151 20.80493
23.83598
non_lung
| 22 16.98591
.6747158 3.164698 15.58276
18.38906
---------+--------------------------------------------------------------------
combined
| 44 19.65318
.6374133 4.228122 18.36772
20.93865
---------+--------------------------------------------------------------------
diff | 5.334545
.9931371
3.330313 7.338777
------------------------------------------------------------------------------
diff = mean(smk_lung) - mean(non_lung) t
= 5.3714
Ho:
diff = 0
degrees of freedom = 42
Ha: diff < 0
Ha: diff != 0
Ha: diff > 0
Pr(T
< t) = 1.0000 Pr(|T| >
|t|) = 0.0000 Pr(T >
t) = 0.0000
This unpaired option is very useful since it enables you to conduct a t-test without additional data manipulation. You may run the .ttest command with the unpaired option to compare two variables, say leukemia and kidney, as independent samples in STATA. In SAS and SPSS, however, you have to stack up two variables and generate a grouping variable before t-tests.
. ttest leukemia=kidney, unpaired
Two-sample t test with equal variances
------------------------------------------------------------------------------
Variable
| Obs Mean
Std. Err. Std. Dev. [95% Conf. Interval]
---------+--------------------------------------------------------------------
leukemia
| 44 6.829773
.0962211 .6382589 6.635724
7.023821
kidney | 44 2.794545
.0782542 .5190799 2.636731
2.95236
---------+--------------------------------------------------------------------
combined
| 88 4.812159
.2249261 2.109994 4.365094
5.259224
---------+--------------------------------------------------------------------
diff | 4.035227
.1240251
3.788673 4.281781
------------------------------------------------------------------------------
diff = mean(leukemia) - mean(kidney)
t = 32.5356
Ho:
diff = 0
degrees of freedom = 86
Ha: diff < 0
Ha: diff != 0
Ha: diff > 0
Pr(T
< t) = 1.0000 Pr(|T| >
|t|) = 0.0000 Pr(T >
t) = 0.0000
The F 1.5119 = (.6532589^2)/(.5190799^2) and its p-value (=.1797) do not reject the null hypothesis of equal variance. The large t statistic 32.5356 rejects the null hypothesis that death rates from leukemia and kidney cancers have the same mean.
4.3 T-test in SAS
The TTEST procedure by default examines the hypothesis of equal variances, and provides T statistics for either case. The procedure by default reports Satterthwaite's approximation for the degrees of freedom. Keep in mind that a variable to be tested is grouped by the variable that is specified in the CLASS statement.
PROC TTEST H0=0 ALPHA=.05 DATA=masil.smoking;
CLASS smoke;
VAR lung;
RUN;
The TTEST Procedure
Statistics
Lower CL Upper CL
Lower CL Upper CL
Variable smoke
N Mean Mean
Mean Std Dev Std Dev Std Dev
lung
0 22 15.583 16.986
18.389 2.4348 3.1647 4.5226
lung
1 22 20.805 22.32
23.836 2.6298 3.4182 4.8848
lung Diff (1-2)
-7.339 -5.335 -3.33 2.7159
3.2939 4.1865
Statistics
Variable smoke Std Err
Minimum Maximum
lung
0 0.6747 12.01
25.45
lung
1 0.7288 12.11
27.27
lung Diff (1-2)
0.9931
T-Tests
Variable Method
Variances DF t Value
Pr > |t|
lung
Pooled Equal
42 -5.37 <.0001
lung Satterthwaite
Unequal 41.8 -5.37
<.0001
Equality of Variances
Variable Method Num DF
Den DF F Value Pr > F
lung
Folded F 21 21
1.17 0.7273
The F test for equal variance does not reject the null hypothesis of equal variances. Thus, the t-test labeled as "Pooled" should be referred to in order to get the t -5.37 and its p-value .0001. If the equal variance assumption is violated, the statistics of "Satterthwaite" and "Cochran" should be read.
If you have a summary data set with the values of variables (lung) and their frequency (count), specify the count variable in the FREQ statement.
PROC TTEST DATA=masil.smoking;
CLASS smoke;
VAR lung;
FREQ count;
RUN;
Now, let us compare the death rates from leukemia and kidney in the second data arrangement type of Figure 1. As mentioned before, you need to rearrange the data set to stack up two variables into one and generate a grouping variable (first type in Figure 1).
DATA masil.smoking2;
SET masil.smoking;
death = leukemia; leu_kid ='Leukemia'; OUTPUT;
death = kidney; leu_kid ='Kidney'; OUTPUT;
KEEP leu_kid death;
RUN;
PROC TTEST COCHRAN DATA=masil.smoking2; CLASS leu_kid; VAR death; RUN;
The TTEST Procedure
Statistics
Lower CL Upper CL
Lower CL Upper CL
Variable
leu_kid N
Mean Mean Mean Std
Dev Std Dev Std Dev Std Err
death
Kidney 44
2.6367 2.7945 2.9524 0.4289
0.5191 0.6577 0.0783
death
Leukemia 44 6.6357
6.8298 7.0238 0.5273 0.6383
0.8087 0.0962
death
Diff (1-2) -4.282
-4.035 -3.789 0.5063 0.5817
0.6838 0.124
T-Tests
Variable Method
Variances DF t Value
Pr > |t|
death Pooled
Equal 86
-32.54 <.0001
death Satterthwaite Unequal
82.6 -32.54 <.0001
death Cochran
Unequal 43
-32.54 <.0001
Equality of Variances
Variable Method Num DF
Den DF F Value Pr > F
death Folded F
43 43
1.51 0.1794
Compare this SAS output with that of STATA in the previous section.
4.4 T-test in SPSS
In the T-TEST command, you need to use the /GROUP subcommand in order to specify a grouping variable. SPSS reports Levene's F .0000 that does not reject the null hypothesis of equal variance p <.995).
T-TEST GROUPS = smoke(0 1)
/VARIABLES = lung
/MISSING = ANALYSIS
/CRITERIA = CI(.95) .
Top
If the assumption of equal variances is violated, we have to compute the adjusted t statistic using individual sample standard deviations rather than a pooled standard deviation. It is also necessary to use the Satterthwaite, Cochran-Cox (SAS), or Welch (STATA) approximations of the degrees of freedom. In this chapter, you compare mean death rates from kidney cancer between the west (south) and east (north).
5.1 T-test in STATA
As discussed earlier, let us check equality of variances using the .oneway command. The tabulate option produces a table of summary statistics for the groups.
. oneway kidney west, tabulate
| Summary of kidney
west | Mean Std. Dev.
Freq.
------------+------------------------------------
0 | 3.006 .3001298
20
1 | 2.6183333 .59837219
24
------------+------------------------------------
Total | 2.7945455 .51907993
44
Analysis of Variance
Source
SS df
MS F
Prob > F
------------------------------------------------------------------------
Between
groups 1.63947758
1 1.63947758 6.92
0.0118
Within
groups 9.94661333 42
.236824127
------------------------------------------------------------------------
Total 11.5860909
43 .269443975
Bartlett's test for equal
variances: chi2(1) = 8.6506 Prob>chi2 = 0.003
Bartlett's chi-squared statistic rejects the null hypothesis of equal variance at the .01 level. It is appropriate to use the unequal option in the .ttest command, which calculates Satterthwaite's approximation for the degrees of freedom.
Unlike the SAS TTEST procedure, the .ttest command cannot specify the mean difference D0 other than zero. Thus, the null hypothesis is that the mean difference is zero.
. ttest kidney, by(west) unequal level(95)
Two-sample
t test with unequal variances
------------------------------------------------------------------------------
Group | Obs
Mean Std. Err. Std. Dev. [95% Conf.
Interval]
---------+--------------------------------------------------------------------
0 | 20 3.006
.0671111 .3001298 2.865535
3.146465
1 | 24 2.618333
.1221422 .5983722 2.365663
2.871004
---------+--------------------------------------------------------------------
combined
| 44 2.794545
.0782542 .5190799 2.636731
2.95236
---------+--------------------------------------------------------------------
diff | .3876667
.139365
.1047722 .6705611
------------------------------------------------------------------------------
diff = mean(0) - mean(1) t
= 2.7817
Ho:
diff = 0
Satterthwaite's degrees of freedom = 35.1098
Ha: diff < 0
Ha: diff != 0
Ha: diff > 0
Pr(T
< t) = 0.9957 Pr(|T| >
|t|) = 0.0086 Pr(T >
t) = 0.0043
See Satterthwaite's approximation of 35.110 in the middle of the output. If you want to get Welch's approximation, use the welch as well as unequal options; without the unequal option, the welch is ignored.
. ttest kidney, by(west) unequal welch
Two-sample
t test with unequal variances
------------------------------------------------------------------------------
Group | Obs
Mean Std. Err. Std. Dev. [95% Conf.
Interval]
---------+--------------------------------------------------------------------
0 | 20 3.006
.0671111 .3001298 2.865535
3.146465
1 | 24 2.618333
.1221422 .5983722 2.365663
2.871004
---------+--------------------------------------------------------------------
combined
| 44 2.794545
.0782542 .5190799 2.636731
2.95236
---------+--------------------------------------------------------------------
diff | .3876667
.139365
.1050824 .6702509
------------------------------------------------------------------------------
diff = mean(0) - mean(1)
t = 2.7817
Ho:
diff = 0
Welch's degrees of freedom = 36.2258
Ha: diff < 0 Ha:
diff != 0
Ha: diff > 0
Pr(T
< t) = 0.9957 Pr(|T| >
|t|) = 0.0085 Pr(T >
t) = 0.0043
Satterthwaite's approximation is slightly smaller than Welch's 36.2258. Again, keep in mind that these approximations are not integers, but real numbers. The t statistic 2.7817 and its p-value .0086 reject the null hypothesis of equal population means. The north and east have larger death rates from kidney cancer per 100 thousand people than the south and west.
For aggregate data, use the .ttesti command with the necessary options.
. ttesti 20 3.006 .3001298 24 2.618333 .5983722, unequal welch
As mentioned earlier, the unpaired option of the .ttest command directly compares two variables without data manipulation. The option treats the two variables as independent of each other. The following is an example of the unpaired and unequal options.
. ttest bladder=kidney, unpaired unequal welch
Two-sample
t test with unequal variances
------------------------------------------------------------------------------
Variable
| Obs Mean
Std. Err. Std. Dev. [95% Conf. Interval]
---------+--------------------------------------------------------------------
bladder
| 44 4.121136
.1454679 .9649249 3.827772
4.4145
kidney | 44 2.794545
.0782542 .5190799 2.636731
2.95236
---------+--------------------------------------------------------------------
combined
| 88 3.457841
.1086268 1.019009 3.241933
3.673748
---------+--------------------------------------------------------------------
diff | 1.326591
.1651806
.9968919 1.65629
------------------------------------------------------------------------------
diff = mean(bladder) - mean(kidney)
t = 8.0312
Ho:
diff = 0
Welch's degrees of freedom = 67.0324
Ha: diff < 0
Ha: diff != 0
Ha: diff > 0
Pr(T
< t) = 1.0000 Pr(|T| >
|t|) = 0.0000 Pr(T >
t) = 0.0000
The F 3.4556 = (.9649249^2)/(.5190799^2) rejects the null hypothesis of equal variance p <0001). If the welch option is omitted, Satterthwaite's degree of freedom 65.9643 will be produced instead.
For aggregate data, again, use the .ttesti command without the unpaired option.
. ttesti 44 4.121136 .9649249 44 2.794545 .5190799, unequal welch level(95)
5.2 T-test in SAS
The TTEST procedure reports statistics for cases of both equal and unequal variance. You may add the COCHRAN option to compute Cochran-Cox approximations for the degree of freedom.
PROC TTEST COCHRAN DATA=masil.smoking;
CLASS west;
VAR kidney;
RUN;
The TTEST Procedure
Statistics
Lower CL Upper CL
Lower CL Upper CL
Variable s_west
N Mean Mean
Mean Std Dev Std Dev Std Dev
kidney
0 20 2.8655 3.006
3.1465 0.2282 0.3001 0.4384
kidney
1 24 2.3657 2.6183
2.871 0.4651 0.5984 0.8394
kidney Diff (1-2)
0.0903 0.3877 0.685 0.4013
0.4866 0.6185
Statistics
Variable
west Std Err Minimum
Maximum
kidney
0 0.0671 2.34
3.62
kidney
1 0.1221 1.59
4.32
kidney Diff (1-2) 0.1473
T-Tests
Variable Method
Variances DF t Value
Pr > |t|
kidney Pooled
Equal 42
2.63 0.0118
kidney Satterthwaite Unequal
35.1 2.78
0.0086
kidney Cochran
Unequal .
2.78 0.0109
Equality of Variances
Variable Method Num DF
Den DF F Value Pr > F
kidney Folded F
23 19
3.97 0.0034
F 3.9749 = (.5983722^2)/(.3001298^2) and p <.0034 reject the null hypothesis of equal variances. Thus, individual sample standard deviations need to be used to compute the adjusted t, and either Satterthwaite's or the Cochran-Cox approximation should be used in computing the p-value. See the following computations.
The t statistic 2.78 rejects the null hypothesis of no difference in mean death rates between the two regions p <.0086).
Now, let us compare death rates from bladder and kidney cancers using SAS.
DATA masil.smoking3;
SET masil.smoking;
death = bladder; bla_kid ='Bladder'; OUTPUT;
death = kidney; bla_kid ='Kidney'; OUTPUT;
KEEP bla_kid death;
RUN;
PROC TTEST COCHRAN DATA=masil.smoking3; CLASS bla_kid; VAR death; RUN;
The TTEST Procedure
Statistics
Lower CL Upper CL
Lower CL Upper CL
Variable
bla_kid N
Mean Mean Mean Std
Dev Std Dev Std Dev Std Err
death
Bladder 44 3.8278
4.1211 4.4145 0.7972 0.9649
1.2226 0.1455
death
Kidney 44
2.6367 2.7945 2.9524 0.4289
0.5191 0.6577 0.0783
death
Diff (1-2) 0.9982
1.3266 1.655 0.6743 0.7748
0.9107 0.1652
T-Tests
Variable Method
Variances DF t Value
Pr > |t|
death Pooled
Equal 86
8.03 <.0001
death Satterthwaite Unequal
66 8.03 <.0001
death Cochran
Unequal 43
8.03 <.0001
Equality of Variances
Variable Method Num DF
Den DF F Value Pr > F
death Folded F
43 43
3.46 <.0001
Fortunately, the t-tests under equal and unequal variance in this case lead the same conclusion at the .01 level; that is, the means of the two death rates are not the same.
5.3 T-test in SPSS
Like SAS, SPSS also reports t statistics for cases of both equal and unequal variance. Note that Levene's F 5.466 rejects the null hypothesis of equal variance at the .05 level p <.024).
T-TEST GROUPS = west(0 1)
/VARIABLES = kidney
/MISSING = ANALYSIS
/CRITERIA = CI(.95) .
Top
The t-test is a special case of one-way ANOVA. Thus, one-way ANOVA produces equivalent results to those of the t-test. ANOVA examines mean differences using the F statistic, whereas the t-test reports the t statistic. The one-way ANOVA (t-test), GLM, and linear regression present essentially the same things in different ways.
6.1 One-way ANOVA
Consider the following ANOVA procedure. The CLASS statement is used to specify categorical variables. The MODEL statement lists the variable to be compared and a grouping variable, separating them with an equal sign.
PROC ANOVA DATA=masil.smoking;
CLASS smoke;
MODEL lung=smoke;
RUN;
The ANOVA Procedure
Dependent
Variable: lung
Sum of
Source
DF Squares
Mean Square F Value Pr > F
Model
1 313.0311273 313.0311273
28.85 <.0001
Error
42 455.6804273 10.8495340
Corrected Total
43 768.7115545
R-Square Coeff Var Root
MSE lung Mean
0.407215 16.75995
3.293863 19.65318
Source
DF Anova SS
Mean Square F Value Pr > F
smoke
1 313.0311273 313.0311273
28.85 <.0001
STATA .anova and .oneway commands also conduct one-way ANOVA.
. anova lung smoke
Number of obs = 44 R-squared
= 0.4072
Root MSE = 3.29386 Adj
R-squared = 0.3931
Source | Partial SS df
MS F
Prob > F
-----------+----------------------------------------------------
Model | 313.031127 1 313.031127
28.85 0.0000
|
smoke | 313.031127 1 313.031127
28.85 0.0000
|
Residual | 455.680427 42 10.849534
-----------+----------------------------------------------------
Total | 768.711555 43 17.8770129
In SPSS, the ONEWAY command is used.
ONEWAY lung BY smoke
/MISSING ANALYSIS .
6.2 Generalized Linear Model (GLM)
The SAS GLM and MIXED procedures and SPSS UNIANOVA command also report the F statistic for one-way ANOVA. Note that STATA's .glm command does not perform one-way ANOVA.
PROC GLM DATA=masil.smoking;
CLASS smoke;
MODEL lung=smoke /SS3;
RUN;
The GLM Procedure
Dependent
Variable: lung
Sum of
Source
DF Squares
Mean Square F Value Pr > F
Model
1 313.0311273
313.0311273 28.85 <.0001
Error
42 455.6804273 10.8495340
Corrected Total
43 768.7115545
R-Square Coeff Var Root
MSE lung Mean
0.407215
16.75995 3.293863
19.65318
Source
DF Type III SS Mean Square
F Value Pr > F
smoke
1 313.0311273 313.0311273
28.85 <.0001
The MIXED procedure has the similar usage as the GLM procedure. The output here is skipped.
PROC MIXED; CLASS smoke; MODEL lung=smoke; RUN;
In SPSS, the UNIANOVA command estimates univariate ANOVA models using the GLM method.
UNIANOVA lung BY smoke
/METHOD = SSTYPE(3)
/INTERCEPT = INCLUDE
/CRITERIA = ALPHA(.05)
/DESIGN = smoke .
6.3 Regression
The SAS REG procedure, STATA .regress command, and SPSS REGRESSION command estimate linear regression models.
PROC REG DATA=masil.smoking;
MODEL lung=smoke;
RUN;
The REG Procedure
Model: MODEL1
Dependent Variable: lung
Number of Observations Read
44
Number
of Observations Used 44
Analysis of Variance
Sum of Mean
Source
DF Squares
Square F Value Pr > F
Model
1 313.03113 313.03113
28.85 <.0001
Error
42 455.68043
10.84953
Corrected Total 43
768.71155
Root MSE
3.29386 R-Square 0.4072
Dependent Mean 19.65318
Adj R-Sq 0.3931
Coeff Var
16.75995
Parameter Estimates
Parameter
Standard
Variable DF Estimate
Error t Value Pr > |t|
Intercept 1 16.98591
0.70225 24.19 <.0001
smoke 1
5.33455 0.99314
5.37 <.0001
Look at the results above. The coefficient of the intercept 16.9859 is the mean of the first group (smoke=0). The coefficient of smoke is, in fact, mean difference between two groups with its sign reversed (5.33455=16.9859-22.3205). Finally, the standard error of the coefficient is the denominator of the independent sample t-test, .99314=3.2939*sqrt(1/22+1/22),where the pooled variance estimate 10.8497=3.2939^2 (see page 11 and 13). Thus, the t 5.37 is identical to the t statistic of the independent sample t-test with equal variance.
The STATA .regress command is quite simple. Note that a dependent variable precedes a list of independent variables.
. regress lung smoke
Source | SS
df MS
Number of obs = 44
-------------+------------------------------
F( 1, 42) = 28.85
Model | 313.031127 1 313.031127
Prob > F = 0.0000
Residual | 455.680427 42 10.849534
R-squared = 0.4072
-------------+------------------------------
Adj R-squared = 0.3931
Total | 768.711555 43 17.8770129
Root MSE = 3.2939
------------------------------------------------------------------------------
lung | Coef. Std. Err.
t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
smoke | 5.334545 .9931371 5.37
0.000 3.330314 7.338777
_cons | 16.98591 .702254 24.19
0.000 15.5687 18.40311
------------------------------------------------------------------------------
The SPSS REGRESSION command looks complicated compared to the SAS REG procedure and STATA .regress command.
REGRESSION
/MISSING LISTWISE
/STATISTICS COEFF OUTS R ANOVA
/CRITERIA=PIN(.05) POUT(.10)
/NOORIGIN
/DEPENDENT lung
/METHOD=ENTER smoke.
Note that ANOVA, GLM, and regression report the same F (1, 42) 28.85, which is equivalent to t (42) -5.3714. As long as the degrees of freedom of the numerator is 1, F is always t^2 (28.85=-5.3714^2).
Top
The t-test is a basic statistical method for examining the mean difference between two groups. One-way ANOVA can compare means of more than two groups. The number of observations in individual groups does not matter in the t-test or one-way ANOVA; both balanced and unbalanced data are fine. One-way ANOVA, GLM, and linear regression models all use the variance-covariance structure in their analysis, but present the results in different ways.
Researchers must check four issues when performing t-tests. First, a variable to be tested should be interval or ratio so that its mean is substantively meaningful. Do not, for example, run a t-test to compare the mean of skin colors (white=0, yellow=1, black=2) between two countries. If you have a latent variable measured by several Likert-scaled manifest variables, first run a factor analysis to get that latent variable.
Second, examine the normality assumptions before conducting a t-test. It is awkward to compare means of variables that are not normally distributed. Figure 2 illustrates a normal probability distribution on top and a Poisson distribution skewed to the right on the bottom. Although the two distributions have the same mean and variance of 1, they are not likely to be substantively interpretable. This is the rationale to conduct normality test such as Shapiro-Wilk W, Shapiro-Francia W, and Kolmogorov-Smirnov D statistics. If the normality assumption is violated, try to use nonparametric methods.
Figure 2. Comparing Normal and Poisson Probability Distributions (sigma^2=1 and mu=1)
Third, check the equal variance assumption. You should be careful when comparing means of normally distributed variables with different variances. You may conduct the folded form F test. If the equal variance assumption is violated, compute the adjusted t and approximations of the degree of freedom.
Finally, consider the types of t-tests, data arrangement, and functionalities available in each statistical software (e.g., STATA, SAS, and SPSS) to determine the best strategy for data analysis (Table 3). The first data arrangement in Figure 1 is commonly used for independent sample t-tests, whereas the second arrangement is appropriate for a paired sample test. Keep in mind that the type II data sets in Figure 1 needs to be reshaped into type I in SAS and SPSS.
Table 3. Comparison of T-test Functionalities of STATA, SAS and SPSS
|
STATA 9.0 |
SAS 9.1 |
SPSS 13.0 |
| Test for equal variance |
Bartlett's chi-squared
(.ttest command) |
Folded form F
(TTEST procedure) |
Levene's weighted F
(T-TEST command) |
| Approximation of degrees of freedom (DF) |
Satterthwaite's DF
Welch's DF |
Satterthwaite's DF
Cochran-Cox DF |
Satterthwaite's DF
|
| Second Data Arrangement |
var1=var2 |
Reshaping the data set |
Reshaping the data set |
| Aggregate Data |
.ttesti command |
FREQ option |
N/A |
SAS has several procedures (e.g., TTEST, MEANS, and UNIVARIATE) and useful options for t-tests. The STATA .ttest and .ttesti commands provide very flexible ways of handling different data arrangements and aggregate data. Table 4 summarizes usages of options in these two commands.
Table 4 Summary of the Usages of the .ttest and .ttesti Command Options
|
Usage |
by (group var) |
unequal |
welch |
unpaired* |
| Univariate sample |
var=c |
| | | |
| Paired (dependent) sample |
var1=var2 |
| | | |
| Equal variance (1 variable) |
var |
O | | | |
| Equal variance (2 variable)** |
var1=var2 |
O | | |
O |
| Unequal variance (1 variable) |
var |
O |
O |
O |
|
| Unequal variance (2 variable) |
var1=var2 |
|
O |
O |
O |
* The .ttesti command does not allow the unpaired option.
** The "var1=var2" assumes second type of data arrangement in Figure 1.
Top
Literature: Fraumeni, J. F. 1968. "Cigarette Smoking and Cancers of the Urinary Tract: Geographic Variations in the United States," Journal of the National Cancer Institute, 41(5): 1205-1211.
Download (UITS Stat/Math Center):
ASCII Text |
STATA |
SAS |
SPSS
The data are per capita numbers of cigarettes smoked (sold) by 43 states and the District of Columbia in 1960 together with death rates per 100 thousand people from various forms of cancer. The variables used in this document are,
- cigar = number of cigarettes smoked (hds per capita)
- bladder = deaths per 100k people from bladder cancer
- lung = deaths per 100k people from lung cancer
- kidney = deaths per 100k people from kidney cancer
- leukemia = deaths per 100k people from leukemia
- smoke = 1 for those whose cigarette consumption is larger than the median and 0 otherwise.
- west = 1 for states in the South or West and 0 for those in the North, East or Midwest.
. sum cigar-leukemia
Variable | Obs
Mean Std. Dev. Min
Max
-------------+-----------------------------------------------------
cigar | 44 24.91409
5.573286 14
42.4
bladder | 44 4.121136
.9649249 2.86
6.54
lung | 44 19.65318
4.228122 12.01 27.27
kidney | 44 2.794545
.5190799 1.59
4.32
leukemia | 44 6.829773
.6382589 4.9
8.28
. sfrancia cigar-leukemia
Shapiro-Francia W' test for normal data
Variable | Obs W'
V' z
Prob>z
-------------+-------------------------------------------------
cigar | 44 0.93061
3.258 2.203 0.01381
bladder | 44 0.94512
2.577 1.776 0.03789
lung | 44 0.97809
1.029 0.055 0.47823
kidney | 44 0.97732
1.065 0.120 0.45217
leukemia | 44 0.97269
1.282 0.474 0.31759
. tab west smoke
| smoke
west |
0 1 |
Total
-----------+----------------------+----------
0 |
7 13 |
20
1 | 15
9 | 24
-----------+----------------------+----------
Total | 22
22 | 44
Top
- Fraumeni, J. F. 1968. "Cigarette Smoking and Cancers of the Urinary Tract: Geographic Variations in the United States," Journal of the National Cancer Institute, 41:5, 1205-1211.
- Ott, R. Lyman. 1993. An Introduction to Statistical Methods and Data Analysis. Belmont, CA: Duxbury Press..
- SAS Institute. 2005. SAS/STAT User's Guide, Version 9.1. Cary, NC: SAS Institute.
- SPSS Inc. 2001. SPSS 11.0 Syntax Reference Guide. Chicago, IL: SPSS Inc.
- STATA Press. 2005. STATA Reference Manual Release 9. College Station, TX: STATA Press.
- Walker, Glenn A. 2002. Common Statistical Methods for Clinical Research with SAS Examples, , NC: SAS Institute.
I am grateful to Jeremy Albright, Takuya Noguchi, and Kevin Wilhite at the Center for Statistical and Mathematical Computing, Indiana University, who provided valuable comments and suggestions.
- 2003. First draft.
- 2004. Second draft.
- 2005. Third draft (Added data arrangements and conclusion).
