Tools needed to assess risk of bias in different reporting scenarios. Note that not all scenarios represent a high risk of bias
| Scenario description | Use tool to assess risk of bias due to missing evidence (eg, ROB-ME)? | Use tool to assess risk of bias in selection of the reported result (eg, RoB 2)? |
|---|---|---|
| Study authors prespecify in ClinicalTrials.gov that pain will be measured yet present no result for pain in any report. | Yes | No |
| Study is listed on a pharmaceutical company’s clinical study register, and several outcomes of interest (eg, depression measured using the Beck Depression Inventory) are prespecified in the abstract for the protocol. However, no results are publicly available, and the request for the required results was rejected by the company. | Yes | No |
| Study authors state in the methods section that health related quality of life was measured, but only state in the results section that there was “no significant difference between groups in quality of life.” Study authors do not respond to requests for fully reported results for this outcome. | Yes | No |
| A journal article published in 2015 describing results of a trial for rheumatoid arthritis has no results for swollen joints. However, measurement of swollen joints is expected to have occurred because it has been deemed a patient important outcome recommended for assessment in all trials since 1994, when it was included in a core outcome set for rheumatoid arthritis. | Yes | No |
| Study authors report in the methods section that they measured depression using three instruments (HAM-D, BDI, and MADRS) at two time points. However, results are reported for BDI at one time point only, and authors provide no further information about results for the other measures of depression. The systematic reviewers are only willing to include results for the HAM-D measure of depression in the meta-analysis, thus there is no result available for inclusion. | Yes | No |
| Study authors report in the methods section that they measured anxiety using three instruments at two time points. However, results are reported for one of the instruments at one time point only, and authors provide no further information about results for the other measures of anxiety. The systematic reviewers are willing to include results for any measure of anxiety in the meta-analysis, thus there is a result available for inclusion. | No | Yes |
| Study authors report that they conducted multiple analyses, each adjusted for different prognostic factors. However, only the unadjusted effect estimate was fully reported, and all adjusted results are referred to only as being “not significant (data not shown).” | No | Yes |
| Study authors prespecify a cut-off point on a continuous measurement scale to create categories of “improved” versus “not improved,” yet the reported result is based on a different cut-off point that was selected after seeing results of the included studies. | No | Yes |
| Study authors report results for active range of motion in flexion in the journal article, yet this outcome was not prespecified in the publicly available trial protocol. | No | Yes |
ROB-ME=tool for assessing risk of bias due to missing evidence; RoB2=tool for assessing risk of bias in randomised trials; HAM-D=Hamilton rating scale for depression; BDI=Beck depression inventory; MADRS=Montgomery-Åsberg depression rating scale.