Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2009 Mar;20(2):416-420.
doi: 10.1093/beheco/arn145. Epub 2008 Nov 27.

Conclusions beyond support: overconfident estimates in mixed models

Holger Schielzeth  1 , Wolfgang Forstmeier
Affiliations

Conclusions beyond support: overconfident estimates in mixed models

Holger Schielzeth et al. Behav Ecol. 2009 Mar.

Abstract

Mixed-effect models are frequently used to control for the nonindependence of data points, for example, when repeated measures from the same individuals are available. The aim of these models is often to estimate fixed effects and to test their significance. This is usually done by including random intercepts, that is, intercepts that are allowed to vary between individuals. The widespread belief is that this controls for all types of pseudoreplication within individuals. Here we show that this is not the case, if the aim is to estimate effects that vary within individuals and individuals differ in their response to these effects. In these cases, random intercept models give overconfident estimates leading to conclusions that are not supported by the data. By allowing individuals to differ in the slopes of their responses, it is possible to account for the nonindependence of data points that pseudoreplicate slope information. Such random slope models give appropriate standard errors and are easily implemented in standard statistical software. Because random slope models are not always used where they are essential, we suspect that many published findings have too narrow confidence intervals and a substantially inflated type I error rate. Besides reducing type I errors, random slope models have the potential to reduce residual variance by accounting for between-individual variation in slopes, which makes it easier to detect treatment effects that are applied between individuals, hence reducing type II errors as well.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Schematic illustrations of more (A) and less (B) problematic cases for the estimation of fixed-effect covariates in random-intercept models. (a) Regression lines for several individuals with high (A) and low (B) between-individual variation in slopes (σb). (b) Two individual regression slopes with low (A) and high (B) scatter around the regression line (σr). (c) Regression lines with (A) many and (B) few measurements per individual (independent of the number of levels of the covariate).
Figure 2
Figure 2
Type I error rate (proportion of estimated 95% confidence intervals for fixed effects not including the true value of zero) in a split-plot design with treatment applied to individuals and slopes measured within individuals as estimated from a random intercept model. The 3 figures in one row show the type I error rates for tests for the treatment main effect (left), the slope main effect (centre), and the slope-by-treatment interaction (right). The 2 rows of figures show simulation with 2 different numbers of measurements within individuals. Type I error rates are indicated by shades of orange (the darker, the higher) and isolines. Error rates depend on the amount of within-individual scatter around the individual regression line (σr, x axis), and the between-individual variation in regression slopes (σb, y axis).
Figure 3
Figure 3
Type I error rates (proportion of estimated 95% confidence intervals for fixed effects not including the true value) in a split-plot design with treatment applied to individuals and slopes measured within individuals as estimated from a random slope model. For details, see Figure 2.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Bates D. lme4: linear mixed-effects models using S4 classes. R package, 0.99875–9. http://r-forge.r-project.org/projects/lme4/ 2007.
    1. Bolund E, Schielzeth H, Forstmeier W Forthcoming. Compensatory investment in zebra finches: females lay larger eggs when paired to sexually unattractive males. Proc R Soc Lond B Biol Sci. doi:10.1098/rspb.2008.1251. - PMC - PubMed
    1. Faraway JJ. Extending the linear model with R. Boca Raton (FL): Chapman & Hall/CRC; 2006.
    1. Galwey NW. Introduction to mixed modelling: beyond regression and analysis of variance. Chichester (UK): John Wiley & Sons; 2006.
    1. Gelman A. Scaling regression inputs by dividing by two standard deviations. Stat Med. 2008;27:2865–2873. - PubMed