Evaluation of excess statistical significance in meta-analyses of 98 biomarker associations with cancer risk
- PMID: 23090067
- DOI: 10.1093/jnci/djs437
Evaluation of excess statistical significance in meta-analyses of 98 biomarker associations with cancer risk
Abstract
Background: Numerous biomarkers have been associated with cancer risk. We assessed whether there is evidence for excess statistical significance in results of cancer biomarker studies, suggesting biases.
Methods: We systematically searched PubMed for meta-analyses of nongenetic biomarkers and cancer risk. The number of observed studies with statistically significant results was compared with the expected number, based on the statistical power of each study under different assumptions for the plausible effect size. We also evaluated small-study effects using asymmetry tests. All statistical tests were two-sided.
Results: We included 98 meta-analyses with 847 studies. Forty-three meta-analyses (44%) found nominally statistically significant summary effects (random effects). The proportion of meta-analyses with statistically significant effects was highest for infectious agents (86%), inflammatory (67%), and insulin-like growth factor (IGF)/insulin system (52%) biomarkers. Overall, 269 (32%) individual studies observed nominally statistically significant results. A statistically significant excess of the observed over the expected number of studies with statistically significant results was seen in 20 meta-analyses. An excess of observed vs expected was observed in studies of IGF/insulin (P ≤ .04) and inflammation systems (P ≤ .02). Only 12 meta-analyses (12%) had a statistically significant summary effect size, more than 1000 case patients, and no hints of small-study effects or excess statistical significance; only four of them had large effect sizes, three of which pertained to infectious agents (Helicobacter pylori, hepatitis and human papilloma viruses).
Conclusions: Most well-documented biomarkers of cancer risk without evidence of bias pertain to infectious agents. Conversely, an excess of statistically significant findings was observed in studies of IGF/insulin and inflammation systems, suggesting reporting biases.
Similar articles
-
An exploratory test for an excess of significant findings.Clin Trials. 2007;4(3):245-53. doi: 10.1177/1740774507079441. Clin Trials. 2007. PMID: 17715249
-
[Meta-analysis of the Italian studies on short-term effects of air pollution].Epidemiol Prev. 2001 Mar-Apr;25(2 Suppl):1-71. Epidemiol Prev. 2001. PMID: 11515188 Italian.
-
Bias in associations of emerging biomarkers with cardiovascular disease.JAMA Intern Med. 2013 Apr 22;173(8):664-71. doi: 10.1001/jamainternmed.2013.3018. JAMA Intern Med. 2013. PMID: 23529078
-
Effect of formal statistical significance on the credibility of observational associations.Am J Epidemiol. 2008 Aug 15;168(4):374-83; discussion 384-90. doi: 10.1093/aje/kwn156. Epub 2008 Jul 8. Am J Epidemiol. 2008. PMID: 18611956 Review.
-
Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects.J Clin Epidemiol. 2011 Oct;64(10):1060-9. doi: 10.1016/j.jclinepi.2010.12.012. Epub 2011 Mar 31. J Clin Epidemiol. 2011. PMID: 21454050 Review.
Cited by
-
Risk factors for preterm birth: an umbrella review of meta-analyses of observational studies.BMC Med. 2023 Dec 13;21(1):494. doi: 10.1186/s12916-023-03171-4. BMC Med. 2023. PMID: 38093369 Free PMC article. Review.
-
Homelessness and health-related outcomes: an umbrella review of observational studies and randomized controlled trials.BMC Med. 2022 Jul 12;20(1):224. doi: 10.1186/s12916-022-02423-z. BMC Med. 2022. PMID: 35818057 Free PMC article.
-
Neurological, Psychiatric, and Psychological Implications of the COVID-19 Pandemic: Protocol for a Large-Scale Umbrella Review of Observational Studies.Int J Environ Res Public Health. 2022 Feb 1;19(3):1681. doi: 10.3390/ijerph19031681. Int J Environ Res Public Health. 2022. PMID: 35162704 Free PMC article.
-
Environmental Risk Factors for Endometriosis: An Umbrella Review of a Meta-Analysis of 354 Observational Studies With Over 5 Million Populations.Front Med (Lausanne). 2021 Oct 25;8:680833. doi: 10.3389/fmed.2021.680833. eCollection 2021. Front Med (Lausanne). 2021. PMID: 34760897 Free PMC article.
-
Associations of Nutritional, Lifestyle, and Metabolic Factors With Non-alcoholic Fatty Liver Disease: An Umbrella Review With More Than 380,000 Participants.Front Nutr. 2021 Sep 17;8:642509. doi: 10.3389/fnut.2021.642509. eCollection 2021. Front Nutr. 2021. PMID: 34604270 Free PMC article.
Publication types
MeSH terms
Substances
LinkOut - more resources
Full Text Sources