Commentaries on Significance Testing
Compiled by David F. Parkhurst, School
of Public and Environmental Affairs, Indiana University,
Bloomington, IN 47405
Not only do we as individuals get locked into single-minded
views, but we also reinforce these views for each other until the culture
itself suffers from the same mindlessness. There is an awareness of this in
science. Scientists proceed along a path gathering data that builds an accepted
wisdom. At some point someone turns everyone's attention to a very different
view of the previously acknowledged truth. This phenomenon happens frequently
enough that scientists are generally not surprised by what is called a paradigm
shift. In a recent New York Times article psychologist Dean Radin described
four stages of adopting ideas: "The first is, 1. 'It's impossible.' 2.
'Maybe it's possible, but it's weak and uninteresting.' 3. 'It is true and I
told you so.' 4. 'I thought of it first.'" I would add a fifth stage,
"We always knew that. How could it be otherwise?" — Ellen J. Langer
(1997), Radin (1996); see References below.
I suspect (and hope) that such a paradigm shift is occurring with the very
way we "build wisdom" from the data we gather, namely, a shift away
from significance testing toward more rational ways of extracting knowledge
from data. I only wish it would occur more quickly than it is.
Comments and suggestions may be sent to firstname.lastname@example.org. Additional
quotes (of all three types) will be considered for inclusion if documented,
with page numbers included.
Keywords: Null hypothesis significance testing, statistical hypothesis
testing, Type I error, Type II error, statistical power.