Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?
- PMID: 9746022
- DOI: 10.1016/S0140-6736(98)01085-X
Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?
Abstract
Background: Few meta-analyses of randomised trials assess the quality of the studies included. Yet there is increasing evidence that trial quality can affect estimates of intervention efficacy. We investigated whether different methods of quality assessment provide different estimates of intervention efficacy evaluated in randomised controlled trials (RCTs).
Methods: We randomly selected 11 meta-analyses that involved 127 RCTs on the efficacy of interventions used for circulatory and digestive diseases, mental health, and pregnancy and childbirth. We replicated all the meta-analyses using published data from the primary studies. The quality of reporting of all 127 clinical trials was assessed by means of component and scale approaches. To explore the effects of quality on the quantitative results, we examined the effects of different methods of incorporating quality scores (sensitivity analysis and quality weights) on the results of the meta-analyses.
Findings: The quality of trials was low. Masked assessments provided significantly higher scores than unmasked assessments (mean 2.74 [SD 1.10] vs 2.55 [1.20]). Low-quality trials (score < or = 2), compared with high-quality trials (score > 2), were associated with an increased estimate of benefit of 34% (ratio of odds ratios [ROR] 0.66 [95% CI 0.52-0.83]). Trials that used inadequate allocation concealment, compared with those that used adequate methods, were also associated with an increased estimate of benefit (37%; ROR=0.63 [0.45-0.88]). The average treatment benefit was 39% (odds ratio [OR] 0.61 [0.57-0.65]) for all trials, 52% (OR 0.48 [0.43-0.54]) for low-quality trials, and 29% (OR 0.71 [0.65-0.77]) for high-quality trials. Use of all the trial scores as quality weights reduced the effects to 35% (OR 0.65 [0.59-0.71]) and resulted in the least statistical heterogeneity.
Interpretation: Studies of low methodological quality in which the estimate of quality is incorporated into the meta-analyses can alter the interpretation of the benefit of intervention, whether a scale or component approach is used in the assessment of trial quality.
Comment in
-
Can quality of clinical trials and meta-analyses be quantified?Lancet. 1998 Aug 22;352(9128):590-1. doi: 10.1016/S0140-6736(98)22034-4. Lancet. 1998. PMID: 9746014 No abstract available.
-
Quality of reports of randomised trials and estimates of treatment efficacy.Lancet. 1998 Nov 7;352(9139):1555-6. doi: 10.1016/s0140-6736(05)60369-8. Lancet. 1998. PMID: 9820334 No abstract available.
Similar articles
-
A meta-epidemiological study to examine the association between bias and treatment effects in neonatal trials.Evid Based Child Health. 2014 Dec;9(4):1052-9. doi: 10.1002/ebch.1985. Evid Based Child Health. 2014. PMID: 25504975
-
Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta-epidemiological studies.Health Technol Assess. 2012 Sep;16(35):1-82. doi: 10.3310/hta16350. Health Technol Assess. 2012. PMID: 22989478 Review.
-
Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiologic study.Ann Intern Med. 2011 Jul 5;155(1):39-51. doi: 10.7326/0003-4819-155-1-201107050-00006. Ann Intern Med. 2011. PMID: 21727292
-
Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study.J Clin Epidemiol. 2011 Oct;64(10):1070-5. doi: 10.1016/j.jclinepi.2010.12.018. Epub 2011 Apr 6. J Clin Epidemiol. 2011. PMID: 21474279 Review.
-
Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses.Ann Intern Med. 2001 Dec 4;135(11):982-9. doi: 10.7326/0003-4819-135-11-200112040-00010. Ann Intern Med. 2001. PMID: 11730399
Cited by
-
Effects of primary or secondary prevention with vitamin A supplementation on clinically important outcomes: a systematic review of randomised clinical trials with meta-analysis and trial sequential analysis.BMJ Open. 2024 May 30;14(5):e078053. doi: 10.1136/bmjopen-2023-078053. BMJ Open. 2024. PMID: 38816049 Free PMC article.
-
Post-thyroidectomy bilateral cervical plexus block relieves pain: a systematic review.Eur Arch Otorhinolaryngol. 2024 Jul;281(7):3765-3778. doi: 10.1007/s00405-024-08626-9. Epub 2024 May 6. Eur Arch Otorhinolaryngol. 2024. PMID: 38709322
-
Antiviral medications for preventing cytomegalovirus disease in solid organ transplant recipients.Cochrane Database Syst Rev. 2024 May 3;5(5):CD003774. doi: 10.1002/14651858.CD003774.pub5. Cochrane Database Syst Rev. 2024. PMID: 38700045 Review.
-
Quality of reporting inflammatory bowel disease randomised controlled trials: a systematic review.BMJ Open Gastroenterol. 2024 Apr 17;11(1):e001337. doi: 10.1136/bmjgast-2023-001337. BMJ Open Gastroenterol. 2024. PMID: 38631808 Free PMC article.
-
Intravenous iron therapy for patients with iron deficiency and heart failure: a systematic review and meta-analysis of randomized controlled trials.Proc (Bayl Univ Med Cent). 2024 Mar 21;37(3):466-476. doi: 10.1080/08998280.2024.2326387. eCollection 2024. Proc (Bayl Univ Med Cent). 2024. PMID: 38628339 Free PMC article.
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources