Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2016 Feb 1;79(3):251-7.
doi: 10.1016/j.biopsych.2015.06.016. Epub 2015 Jun 24.

Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies

Hasse Walum  1 , Irwin D Waldman  2 , Larry J Young  3
Affiliations
Review

Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies

Hasse Walum et al. Biol Psychiatry. .

Abstract

Over the last decade, oxytocin (OT) has received focus in numerous studies associating intranasal administration of this peptide with various aspects of human social behavior. These studies in humans are inspired by animal research, especially in rodents, showing that central manipulations of the OT system affect behavioral phenotypes related to social cognition, including parental behavior, social bonding, and individual recognition. Taken together, these studies in humans appear to provide compelling, but sometimes bewildering, evidence for the role of OT in influencing a vast array of complex social cognitive processes in humans. In this article, we investigate to what extent the human intranasal OT literature lends support to the hypothesis that intranasal OT consistently influences a wide spectrum of social behavior in humans. We do this by considering statistical features of studies within this field, including factors like statistical power, prestudy odds, and bias. Our conclusion is that intranasal OT studies are generally underpowered and that there is a high probability that most of the published intranasal OT findings do not represent true effects. Thus, the remarkable reports that intranasal OT influences a large number of human social behaviors should be viewed with healthy skepticism, and we make recommendations to improve the reliability of human OT studies in the future.

Keywords: Bias; Effect size; Neuroendocrinology; Positive predictive value; Social cognition; Statistical power.

PubMed Disclaimer

Conflict of interest statement

Conflicts of interest

LJY has applied for a patent (US20120108510 - Methods of improving behavioral therapies) for combining melanocortin agonists with behavioral therapies to enhance social cognition in psychiatric disorders. HW and IW declare no biomedical financial interests or potential conflicts of interest.

Figures

Figure 1
Figure 1. Statistical power as a function of effect size and sample size
The figure shows the relationship between sample size and statistical power for four different effect sizes. Power calculations were performed using simulations in R (3.1.1). In the simulations, half of the sample was drawn from a standard normal distribution and the other half from a second normal distribution with a mean representing the investigated effect size. This procedure was repeated 1000 times per effect size and sample size. Power was determined as the proportion of these 1000 “experiments” rejecting the null hypothesis (using one-way ANOVA), with the alpha level set to 0.05. The effects sizes presented in the figure represent the largest (d=.48) and smallest (d=.21) effects sizes within the field of intranasal oxytocin studies in humans, as well as the mean effect size for healthy subjects (d=.28) and clinical trials (d=.32). It is clear that the studies within this field are underpowered since for all effect sizes and sample sizes the statistical power is below 80%, the standard for minimal adequate statistical power.
Figure 2
Figure 2. Effect size inflation as a function of statistical power
Effect size inflation is expected to occur when findings need to pass a certain threshold, in this case statistical significance, in order to be considered positive. The smaller the proportion of effects that pass this threshold, the larger the average effect size inflation will be. Since power is an estimate of the proportion of investigated effects that are statistically significant, low power is associated large effect size inflation. The figure shows simulations of effect size inflation. To generate this data we ran simulations in R (3.1.1) (13), using a similar approach as for the power simulations presented in Figure 1. Here, the amount of inflation was calculated by subtracting the true population effect size from the observed effect size of simulations reaching statistical significance (p<0.05 using one-way ANOVA), and dividing the difference by the true effect size. This estimate was then averaged over 1000 replicates. Due to the low power of IN-OT studies reported effect sizes within this field are generally inflated to a large degree.
Figure 3
Figure 3. Positive predictive value as a function of pre study odds and statistical power
The probability of a research finding representing a true effect (the positive predictive value) is dependent on the pre study odds of an effect being true (R) and statistical power. The figure shows the relationship between pre study odds and positive predictive value, for the average statistical power in IN-OT studies in healthy subjects (16%) and clinical trials (12%), as well as the standard for minimal adequate statistical power (80%). Clearly, compared to adequately powered studies the probability of a research finding reflecting a true effect is strongly reduced for studies with 12% and 16% power, especially when the pre study odds are low.
Figure 4
Figure 4. Positive predictive value as a function of multiple comparisons
The odds of a research finding representing a true effect (the positive predictive value) are reduced as the number of uncorrected tests within a study increases. The figure is showing how multiple comparisons without correction influence the positive predictive value for studies with 12% and 16% power.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Bartz JA, Zaki J, Bolger N, Ochsner KN. Social effects of oxytocin in humans: context and person matter. Trends in cognitive sciences. 2011;15:301–309. - PubMed
    1. Churchland PS, Winkielman P. Modulating social behavior with oxytocin: how does it work? What does it mean? Hormones and behavior. 2012;61:392–399. - PMC - PubMed
    1. Evans SL, Monte OD, Noble P, Averbeck BB. Intranasal oxytocin effects on social cognition: A critique. Brain research 2013 - PMC - PubMed
    1. Guastella AJ, Hickie IB, McGuinness MM, Otis M, Woods EA, Disinger HM, et al. Recommendations for the standardisation of oxytocin nasal administration and guidelines for its reporting in human research. Psychoneuroendocrinology. 2013;38:612–625. - PubMed
    1. Ross HE, Young LJ. Oxytocin and the neural mechanisms regulating social cognition and affiliative behavior. Frontiers in neuroendocrinology. 2009;30:534–547. - PMC - PubMed