**An
open letter to Editors of journals, Chairs of departments, Directors of funding
programs, Directors of graduate training, Reviewers of grants and manuscripts,
Researchers, Teachers, and Students:**

*Statistical methods have been evolving
rapidly, and many people think it’s time to adopt modern Bayesian data analysis
as standard procedure in our scientific practice and in our educational
curriculum. Three reasons:*

*1. Scientific
disciplines from astronomy to zoology are moving to Bayesian data analysis. We
should be leaders of the move, not followers.*

*2. Modern Bayesian
methods provide richer information, with greater flexibility and broader
applicability than 20th century methods. Bayesian methods are intellectually
coherent and intuitive. Bayesian analyses are readily computed with modern
software and hardware.*

*3. Null-hypothesis significance
testing (NHST), with its reliance on p
values, has many problems. There is little reason to persist with NHST now that
Bayesian methods are accessible to everyone.*

*My
conclusion from those points is that we should do whatever we can to encourage
the move to Bayesian data analysis. Journal editors could accept Bayesian data analyses,
and encourage submissions with Bayesian data analyses. Department chairpersons could
encourage their faculty to be leaders of the move to modern Bayesian methods. Funding
agency directors could encourage applications using Bayesian data analysis. Reviewers
could recommend Bayesian data analyses. Directors of training
or curriculum could get courses in Bayesian data analysis incorporated into the
standard curriculum. Teachers can teach Bayesian. Researchers can use Bayesian
methods to analyze data and submit the analyses for publication. Students can get
an advantage by learning and using Bayesian data analysis.*

The goal is encouragement of Bayesian methods,
not prohibition of NHST or other methods. Researchers will embrace Bayesian analysis
once they learn about it and see its many practical and intellectual advantages.
Nevertheless, change requires vision, courage, incentive, effort, and
encouragement!

Now to expand on the three reasons stated
above.

*1.
**Scientific disciplines from astronomy to zoology are
moving to Bayesian data analysis. We should be leaders of the move, not
followers.*

Bayesian methods are revolutionizing science.
Notice the titles of these articles:

**Bayesian
computation: a statistical revolution.**
Brooks, S.P. Philosophical Transactions of the Royal Society of London. Series
A: Mathematical, Physical and Engineering Sciences, 361(1813), 2681, 2003.

**The
Bayesian revolution in genetics.** Beaumont, M.A. and Rannala, B. Nature
Reviews Genetics, 5(4), 251-261, 2004.

**A
Bayesian revolution in spectral analysis.** Gregory, PC. AIP Conference
Proceedings, 557-568, 2001.

**The hierarchical
Bayesian revolution: how Bayesian methods have changed the face of marketing
research**. Allenby, G.M. and Bakken, D.G. and Rossi, P.E. Marketing Research, 16, 20-25,
2004

**The future of
statistics: A Bayesian 21st century.**
Lindley, DV. Advances in Applied Probability, 7, 106-115,
1975.

There are many other articles that make analogous
points in other fields, but with less pithy titles. If nothing else, the titles
above suggest that the phrase “Bayesian revolution” is not an overstatement.

The Bayesian revolution spans many fields of
science. Notice the titles of these articles:

**Bayesian
analysis of hierarchical models and its application in Agriculture.** Nazir, N., Khan, A.A., Shafi, S.,
Rashid, A. InterStat, 1, 2009.

**The Bayesian
approach to the interpretation of archaeological
data**. Litton,
CD & Buck, CE. Archaeometry, 37(1), 1-24, 1995.

**The promise of
Bayesian inference for astrophysics**. Loredo TJ. In: Feigelson ED, Babu GJ, eds.
Statistical Challenges in Modern Astronomy. New York: Springer-Verlag; 1992, 275–297.

**Bayesian methods in
the atmospheric sciences**. Berliner LM, Royle JA, Wikle CK, Milliff RF. In: Bernardo JM, Berger JO, Dawid AP, Smith AFM, eds. Bayesian Statistics 6:
Proceedings of the sixth Valencia international meeting, June 6–10, 1998.
Oxford, UK: Oxford University Press; 1999, 83–100.

**An introduction to
Bayesian methods for analyzing chemistry
data:: Part II: A review of applications of Bayesian
methods in chemistry**. Hibbert, DB and Armstrong, N. Chemometrics and
Intelligent Laboratory Systems, 97(2), 211-220, 2009.

**Bayesian methods in conservation biology**. Wade PR. Conservation Biology, 2000, 1308–1316.

**Bayesian inference
in ecology**. Ellison AM. Ecol Biol 2004, 7:509–520.

**The
Bayesian approach to research in economic
education.**** **Kennedy, P. Journal
of Economic Education, 17, 9-24, 1986.

**The
growth of Bayesian methods in statistics and economics
since 1970**. Poirier, D.J. Bayesian Analysis, 1(4),
969-980, 2006.

**Commentary:
Practical advantages of Bayesian analysis of epidemiologic
data**. Dunson
DB. Am J Epidemiol 2001,
153:1222–1226.

**Bayesian inference
of phylogeny and its impact on evolutionary
biology**. Huelsenbeck
JP, Ronquist F, Nielsen R, Bollback
JP. Science 2001, 294:2310–2314.

**Geoadditive**** Bayesian models for
forestry defoliation data: a case
study. **Musio, M. and Augustin,
N.H. and von Wilpert, K. Environmetrics. 19(6), 630—642, 2008.

**Bayesian statistics
in genetics: a guide for the
uninitiated.** Shoemaker, J.S. and
Painter, I.S. and Weir, B.S. Trends in Genetics, 15(9), 354-358, 1999.

**Bayesian
statistics in oncology.**** **Adamina, M. and Tomlinson, G. and Guller, U. Cancer, 115(23), 5371-5381, 2009.

**Bayesian
analysis in plant pathology.** Mila, AL and Carriquiry,
AL. Phytopathology, 94(9), 1027-1030, 2004.

**Bayesian analysis
for political research**. Jackman S. Annual Review of
Political Science, 2004, 7:483–505.

The list above could go on and on. The point
is simple: Bayesian methods are being adopted across the disciplines of
science. We should not be laggards in utilizing Bayesian methods in our
science, or in teaching Bayesian methods in our classrooms.

Why are Bayesian methods being adopted across
science? Answer:

*2.
**Bayesian methods provide richer information, with greater
flexibility and broader applicability than 20th century methods. Bayesian
methods are intellectually coherent and intuitive. Bayesian analyses are
readily computed with modern software and hardware.*

To explain this point adequately would take
an entire textbook, but here are a few highlights.

* In NHST, the data collector must pretend to
plan the sample size in advance and pretend not to let preliminary looks at the
data influence the final sample size. Bayesian design, on the contrary, has no
such pretenses because inference is not based on *p* values.

* In NHST, analysis of variance (ANOVA) has
elaborate corrections for multiple comparisons based on the intentions of the
analyst. Hierarchical Bayesian ANOVA uses no such corrections, instead
rationally mitigating false alarms based on the data.

* Bayesian computational practice allows easy
modification of models to properly accommodate the measurement scales and
distributional needs of observed data.

* In many NHST analyses, missing data or
otherwise unbalanced designs can produce computational problems. Bayesian
models seamlessly handle unbalanced and small-sample designs.

* In many NHST analyses, individual
differences are challenging to incorporate into the analysis. In hierarchical
Bayesian approaches, individual differences can be flexibly and easily modeled,
with hierarchical priors that provide rational “shrinkage” of individual
estimates.

* In contingency table analysis, the
traditional chi-square test suffers if expected values of cell frequencies are
less than 5. There is no such issue in Bayesian analysis, which handles small or
large frequencies seamlessly.

* In multiple regression analysis,
traditional analyses break down when the predictors are perfectly (or very
strongly) correlated, but Bayesian analysis proceeds as usual and reveals that
the estimated regression coefficients are (anti-)correlated.

* In NHST, the power of an experiment, i.e.,
the probability of rejecting the null hypothesis, is
based on a single alternative hypothesis. And the probability of replicating a
significant outcome is “virtually unknowable” according to recent research. But
in Bayesian analysis, both power and replication probability can be computed in
straight forward manner, with the uncertainty of the hypothesis directly
represented.

* Bayesian computational practice allows easy
specification of domain-specific psychometric models in addition to generic
models such as ANOVA and regression.

Some people may have the mistaken impression
that the advantages of Bayesian methods are negated by the need to specify a prior
distribution. In fact, the use of a prior is both appropriate for rational
inference and advantageous in practical applications.

* It is *in*appropriate
*not* to use a prior. Consider the well
known example of random disease screening. A person is selected at random to be
tested for a rare disease. The test result is positive. What is the probability
that the person actually has the disease? It turns out, even if the test is
highly accurate, the posterior probability of actually having the disease is
surprisingly small. Why? Because the prior probability of the
disease was so small. Thus, incorporating the prior is crucial for
coming to the right conclusion.

* Priors are explicitly specified and must be
agreeable to a skeptical scientific audience. Priors are not capricious and
cannot be covertly manipulated to predetermine a conclusion. If skeptics
disagree with the specification of the prior, then the robustness of the conclusion
can be explicitly examined by considering other reasonable priors. In most
applications, with moderately large data sets and reasonably informed priors,
the conclusions are quite robust.

* Priors are useful for cumulative scientific
knowledge and for leveraging inference from small-sample research. As an
empirical domain matures, more and more data accumulate regarding particular
procedures and outcomes. The accumulated results can inform the priors of
subsequent research, yielding greater precision and firmer conclusions.

* When different groups of scientists have
differing priors, stemming from differing theories and empirical emphases, then
Bayesian methods provide rational means for comparing the conclusions from the
different priors.

To summarize, priors are not a problematic
nuisance to be avoided. Instead, priors should be embraced as appropriate in rational
inference and advantageous in real research.

If those advantages of Bayesian methods are
not enough to attract change, there is also a major reason to be repelled from
the dominant method of the 20th century:

*3.
**20th century null-hypothesis significance testing (NHST),
with its reliance on p values, has many severe problems. There is little reason
to persist with NHST now that Bayesian methods are accessible to everyone.*

Although there are many difficulties in using
*p* values, the fundamental fatal flaw
of *p* values is that they are ill
defined, because any set of data has many different *p* values.

Consider the simple case of assessing whether
an electorate prefers candidate A over candidate B. A quick random poll reveals
that 8 people prefer candidate A out of 23 respondents. What is the *p* value of that outcome if the
population were equally divided? There is no single answer! If the pollster
intended to stop when N=23, then the *p*
value is based on repeating an experiment in which N is fixed at 23. If the
pollster intended to stop after the 8th respondent who preferred candidate A, then the *p*
value is based on repeating an experiment in which N can be anything from 8 to
infinity. If the pollster intended to poll for one hour, then the *p* value is based on repeating an
experiment in which N can be anything from zero to infinity. There is a
different *p* value for every possible
intention of the pollster, even though the observed data are fixed, and even
though the outcomes of the queries are carefully insulated from the intentions
of the pollster.

The problem of ill-defined *p* values is magnified for realistic
situations. In particular, consider the well-known issue of multiple
comparisons in analysis of variance (ANOVA). When there are several groups, we
usually are interested in a variety of comparisons among them: Is group A significantly different from group B? Is group C different
from group D? Is the average of groups A and B different from the average of
groups C and D? Every comparison presents another opportunity for a false
alarm, i.e., rejecting the null hypothesis when it is true. Therefore the NHST
literature is replete with recommendations for how to mitigate the “experimentwise” false alarm rate, using corrections such as
Bonferroni, Tukey, Scheffe, etc. The bizarre part of this practice is that the
*p* value for the single comparison of
groups A and B depends on what other groups you intend to compare them with.
The data in groups A and B are fixed, but merely intending to compare them with
other groups enlarges the *p* value of
the A vs B comparison. The *p* value grows because there is a different space of possible
experimental outcomes when the intended experiment comprises more groups.
Therefore it is trivial to make any comparison have a large *p* value and be nonsignificant;
all you have to do is intend to compare the data with other groups in the
future.

The literature is full of articles pointing
out the many conceptual misunderstandings held by practitioners of NHST. For
example, many people mistake the *p*
value for the probability that the null hypothesis is true. Even if those
misunderstandings could be eradicated, such that everyone clearly understood
what *p* values really are, the *p* values would still be ill defined.
Every fixed set of data would still have many different *p* values.

To recapitulate: Science is moving to
Bayesian methods because of their many advantages, both practical and
intellectual, over 20th century NHST. It is time that we convert our research
and educational practices to Bayesian data analysis. I hope you will encourage
the change. It’s the right thing to do.

John K. Kruschke, Revised
14 November 2010, http://www.indiana.edu/~kruschke/