Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Jul 29:5:781.
doi: 10.3389/fpsyg.2014.00781. eCollection 2014.

Using Bayes to get the most out of non-significant results

Zoltan Dienes  1
Affiliations

Using Bayes to get the most out of non-significant results

Zoltan Dienes. Front Psychol. .

Abstract

No scientific conclusion follows automatically from a statistically non-significant result, yet people routinely use non-significant results to guide conclusions about the status of theories (or the effectiveness of practices). To know whether a non-significant result counts against a theory, or if it just indicates data insensitivity, researchers must use one of: power, intervals (such as confidence or credibility intervals), or else an indicator of the relative evidence for one theory over another, such as a Bayes factor. I argue Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches. Specifically, Bayes factors use the data themselves to determine their sensitivity in distinguishing theories (unlike power), and they make use of those aspects of a theory's predictions that are often easiest to specify (unlike power and intervals, which require specifying the minimal interesting value in order to address theory). Bayes factors provide a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive. They allow accepting and rejecting the null hypothesis to be put on an equal footing. Concrete examples are provided to indicate the range of application of a simple online Bayes calculator, which reveal both the strengths and weaknesses of Bayes factors.

Keywords: Bayes factor; confidence interval; highest density region; null hypothesis; power; significance testing; statistical inference.

PubMed Disclaimer

Figures

FIGURE 1
FIGURE 1
Dance of the p-values. A sequence of p-values (and confidence intervals) for successive simulations of an experiment where there is an actual effect (mean population difference = 10 units) and power is 0.5. Created from software provided by Cumming (2011; associated website).
FIGURE 2
FIGURE 2
The four principles of inference by intervals. (i) If the interval is completely contained in the null region, decide that the population value lies in the null region (accept the null region hypothesis); (ii) If the interval is completely outside the null region, decide that the population value lies outside the null region (reject the null region hypothesis); (iii) If the upper limit of the interval is below the minimal interesting value, decide against a theory postulating a positive difference (reject a directional theory); (iv) If the interval includes both null region and theoretically interesting values, the data are insensitive (suspend judgment).
FIGURE 3
FIGURE 3
Representing the alternative hypothesis. (A) A uniform distribution with all population parameter values from the lower to the upper limit equally plausible. Here the lower limit is 0, a typical but not required value. (B) A normal distribution, with population parameter values close to the mean being more plausible than others. The SD also needs to be specified; a default of mean/2 is often useful. (C) A half-normal distribution. Values close to 0 are most plausible; a useful default for the SD is a typical estimated effect size. Population values less than 0 are ruled out.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Allen C. P. G., Sumner P., Chambers C. D. (2014). The timing and neuroanatomy of conscious vision as revealed by TMS-induced blindsight. J. Cogn. Neurosci. 26 1507–1518 10.1162/jocn_a_00557 - DOI - PubMed
    1. American Psychological Association [APA] (2010). Publication Manual of the American Psychological Association 5th Edn. Washington, DC: AP.
    1. Armitage P., McPherson C. K., Rowe B. C. (1969). Repeated significance tests on accumulating data. J. R. Stat. Soc. Ser. A 132 235–244 10.2307/2343787 - DOI
    1. Armstrong A. M., Dienes Z. (2013). Subliminal understanding of negation: unconscious control by subliminal processing of word pairs. Conscious. Cogn. 22 1022–1040 10.1016/j.concog.2013.06.010 - DOI - PubMed
    1. Baguley T. (2004). Understanding statistical power in the context of applied research. Appl. Ergon. 35 73–80 10.1016/j.apergo.2004.01.002 - DOI - PubMed