Sample-Size Planning for More Accurate Statistical Power: A Method Adjusting Sample Effect Sizes for Publication Bias and Uncertainty
- PMID: 28902575
- DOI: 10.1177/0956797617723724
Sample-Size Planning for More Accurate Statistical Power: A Method Adjusting Sample Effect Sizes for Publication Bias and Uncertainty
Abstract
The sample size necessary to obtain a desired level of statistical power depends in part on the population value of the effect size, which is, by definition, unknown. A common approach to sample-size planning uses the sample effect size from a prior study as an estimate of the population value of the effect to be detected in the future study. Although this strategy is intuitively appealing, effect-size estimates, taken at face value, are typically not accurate estimates of the population effect size because of publication bias and uncertainty. We show that the use of this approach often results in underpowered studies, sometimes to an alarming degree. We present an alternative approach that adjusts sample effect sizes for bias and uncertainty, and we demonstrate its effectiveness for several experimental designs. Furthermore, we discuss an open-source R package, BUCSS, and user-friendly Web applications that we have made available to researchers so that they can easily implement our suggested methods.
Keywords: effect size; methodology; publication bias; sample size; statistical power.
Similar articles
-
Using prior information to plan appropriately powered regression studies: A tutorial using BUCSS.Psychol Methods. 2021 Oct;26(5):513-526. doi: 10.1037/met0000366. Epub 2020 Oct 29. Psychol Methods. 2021. PMID: 33119336
-
Sample Size Planning for Detecting Mediation Effects: A Power Analysis Procedure Considering Uncertainty in Effect Size Estimates.Multivariate Behav Res. 2019 Nov-Dec;54(6):822-839. doi: 10.1080/00273171.2019.1593814. Epub 2019 Apr 15. Multivariate Behav Res. 2019. PMID: 30983425
-
Planning sample sizes when effect sizes are uncertain: The power-calibrated effect size approach.Psychol Methods. 2016 Mar;21(1):47-60. doi: 10.1037/met0000036. Epub 2015 Dec 14. Psychol Methods. 2016. PMID: 26651984
-
On sample size estimation and re-estimation adjusting for variability in confirmatory trials.J Biopharm Stat. 2016;26(1):44-54. doi: 10.1080/10543406.2015.1092031. J Biopharm Stat. 2016. PMID: 26378970 Review.
-
On effect size.Psychol Methods. 2012 Jun;17(2):137-52. doi: 10.1037/a0028086. Epub 2012 Apr 30. Psychol Methods. 2012. PMID: 22545595 Review.
Cited by
-
Further Examination of the Pulsed- and Steady-Pedestal Paradigms under Hypothetical Parvocellular- and Magnocellular-Biased Conditions.Vision (Basel). 2024 Apr 30;8(2):28. doi: 10.3390/vision8020028. Vision (Basel). 2024. PMID: 38804349 Free PMC article.
-
Memory retrieval effects as a function of differences in phenomenal experience.Brain Imaging Behav. 2024 May 6. doi: 10.1007/s11682-024-00892-9. Online ahead of print. Brain Imaging Behav. 2024. PMID: 38709432
-
Uncovering the interplay between drawings, mental representations, and arithmetic problem-solving strategies in children and adults.Mem Cognit. 2024 Feb 12. doi: 10.3758/s13421-024-01523-w. Online ahead of print. Mem Cognit. 2024. PMID: 38347259
-
Statistical power analysis and sample size planning for moderated mediation models.Behav Res Methods. 2024 Feb 2. doi: 10.3758/s13428-024-02342-2. Online ahead of print. Behav Res Methods. 2024. PMID: 38308148
-
Object geometry serves humans' intuitive physics of stability.Sci Rep. 2024 Jan 19;14(1):1701. doi: 10.1038/s41598-024-51677-5. Sci Rep. 2024. PMID: 38242998 Free PMC article.
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources
