Statistical Analysis of Experimental Results

• Goal: to show that the treatment makes a difference
• Generalizing to a population, for example, all people, all adult Americans, all college students, from a representative sample

  

• Assume one independent variable with 2 levels. Our hypothesis: the independent variable will make a difference.
• The null hypothesis: the independent variable does not make a difference. Goal: reject the null hypothesis (claim statistically significant results)

  

• Failing to reject the null hypothesis: failing to show the independent variable matters (it actually may)
• Possible wrong conclusions based on an experiment
  • Incorrectly rejecting the null hypothesis: believing that the treatment makes a difference when it actually doesn't (Type I error)
  • Incorrectly failing to reject the null hypothesis: believing that the treatment doesn't make a difference when it actually does (Type II error)
• Significance level: the probability of incorrectly rejecting the null hypothesis (a Type I error), usually .05 or .01
• What we get from the experiment: two sets of values for the dependent variable, one for each level of the independent variable
• Descriptive statistics
  • Mean: the average of a set of values

    

    We are interested in the overall mean and the mean for each group.

  • Standard deviation: a measure of amount of deviation of the values from the mean

    

• Statistical inference
  • Is the difference in the means for the two groups large enough to claim statistical significance?

    Is the difference in the means for the two groups large enough for us to say that the probability of a Type I error is less than the signifance level that we picked?

  • Statistical tests (for example, t-test)
    • Calculate the value of the statistic for our data
    • Compare that value to the critical value, the value that would be needed to claim statistical significance
    • If our value is greater than the critical value, claim statistical significance
    a particular value for our data is calculated and compared with the critical value, the value that would be needed to claim statistical significance
  • Statistical significance depends on
    • The difference in the group means
    • Standard deviations of the individual subjects' scores
    • Number of subjects
    • Significance level
  • Some possible outcomes

    

    

    

Take me back to the Rhythm and Cognition Home Page.

Last updated: 31 October 1995
URL: http://www.indiana.edu/~gasser/statistics.html
Comments: gasser@salsa.indiana.edu
Copyright 1995, The Trustees of Indiana University