Estimating the mean effect size in meta-analysis: bias, precision, and mean squared error of different weighting methods
- PMID: 14748494
- DOI: 10.3758/bf03195529
Estimating the mean effect size in meta-analysis: bias, precision, and mean squared error of different weighting methods
Abstract
Although use of the standardized mean difference in meta-analysis is appealing for several reasons, there are some drawbacks. In this article, we focus on the following problem: that a precision-weighted mean of the observed effect sizes results in a biased estimate of the mean standardized mean difference. This bias is due to the fact that the weight given to an observed effect size depends on this observed effect size. In order to eliminate the bias, Hedges and Olkin (1985) proposed using the mean effect size estimate to calculate the weights. In the article, we propose a third alternative for calculating the weights: using empirical Bayes estimates of the effect sizes. In a simulation study, these three approaches are compared. The mean squared error (MSE) is used as the criterion by which to evaluate the resulting estimates of the mean effect size. For a meta-analytic dataset with a small number of studies, the MSE is usually smallest when the ordinary procedure is used, whereas for a moderate or large number of studies, the procedures yielding the best results are the empirical Bayes procedure and the procedure of Hedges and Olkin, respectively.
Similar articles
-
Bias caused by sampling error in meta-analysis with small sample sizes.PLoS One. 2018 Sep 13;13(9):e0204056. doi: 10.1371/journal.pone.0204056. eCollection 2018. PLoS One. 2018. PMID: 30212588 Free PMC article.
-
Adjusting for Publication Bias in Meta-Analysis: An Evaluation of Selection Methods and Some Cautionary Notes.Perspect Psychol Sci. 2016 Sep;11(5):730-749. doi: 10.1177/1745691616662243. Perspect Psychol Sci. 2016. PMID: 27694467 Review.
-
A Bayesian nonparametric meta-analysis model.Res Synth Methods. 2015 Mar;6(1):28-44. doi: 10.1002/jrsm.1117. Epub 2014 Apr 29. Res Synth Methods. 2015. PMID: 26035468 Review.
-
Weighting strategies in the meta-analysis of single-case studies.Behav Res Methods. 2014 Dec;46(4):1152-66. doi: 10.3758/s13428-013-0440-0. Behav Res Methods. 2014. PMID: 24488814
-
Averaging dependent effect sizes in meta-analysis: a cautionary note about procedures.Span J Psychol. 1999 May;2(1):32-8. doi: 10.1017/s1138741600005436. Span J Psychol. 1999. PMID: 11757259
Cited by
-
The analysis of continuous data from n-of-1 trials using paired cycles: a simple tutorial.Trials. 2024 Feb 16;25(1):128. doi: 10.1186/s13063-024-07964-7. Trials. 2024. PMID: 38365817 Free PMC article.
-
Cytokine alterations in pediatric internalizing disorders: Systematic review and exploratory multi-variate meta-analysis.Brain Behav Immun Health. 2022 Jul 16;24:100490. doi: 10.1016/j.bbih.2022.100490. eCollection 2022 Oct. Brain Behav Immun Health. 2022. PMID: 35880170 Free PMC article.
-
Evaluation of various estimators for standardized mean difference in meta-analysis.Stat Med. 2021 Jan 30;40(2):403-426. doi: 10.1002/sim.8781. Epub 2020 Nov 12. Stat Med. 2021. PMID: 33180373 Free PMC article.
-
When continuous outcomes are measured using different scales: guide for meta-analysis and interpretation.BMJ. 2019 Jan 22;364:k4817. doi: 10.1136/bmj.k4817. BMJ. 2019. PMID: 30670455 Free PMC article.
-
Do clinicians understand the size of treatment effects? A randomized survey across 8 countries.CMAJ. 2016 Jan 5;188(1):25-32. doi: 10.1503/cmaj.150430. Epub 2015 Oct 26. CMAJ. 2016. PMID: 26504102 Free PMC article. Clinical Trial.