Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review
- PMID: 37266983
- PMCID: PMC10276326
- DOI: 10.2196/43803
Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review
Abstract
Background: In the context of a deepening global shortage of health workers and, in particular, the COVID-19 pandemic, there is growing international interest in, and use of, online symptom checkers (OSCs). However, the evidence surrounding the triage and diagnostic accuracy of these tools remains inconclusive.
Objective: This systematic review aimed to summarize the existing peer-reviewed literature evaluating the triage accuracy (directing users to appropriate services based on their presenting symptoms) and diagnostic accuracy of OSCs aimed at lay users for general health concerns.
Methods: Searches were conducted in MEDLINE, Embase, CINAHL, Health Management Information Consortium (HMIC), and Web of Science, as well as the citations of the studies selected for full-text screening. We included peer-reviewed studies published in English between January 1, 2010, and February 16, 2022, with a controlled and quantitative assessment of either or both triage and diagnostic accuracy of OSCs directed at lay users. We excluded tools supporting health care professionals, as well as disease- or specialty-specific OSCs. Screening and data extraction were carried out independently by 2 reviewers for each study. We performed a descriptive narrative synthesis.
Results: A total of 21,296 studies were identified, of which 14 (0.07%) were included. The included studies used clinical vignettes, medical records, or direct input by patients. Of the 14 studies, 6 (43%) reported on triage and diagnostic accuracy, 7 (50%) focused on triage accuracy, and 1 (7%) focused on diagnostic accuracy. These outcomes were assessed based on the diagnostic and triage recommendations attached to the vignette in the case of vignette studies or on those provided by nurses or general practitioners, including through face-to-face and telephone consultations. Both diagnostic accuracy and triage accuracy varied greatly among OSCs. Overall diagnostic accuracy was deemed to be low and was almost always lower than that of the comparator. Similarly, most of the studies (9/13, 69 %) showed suboptimal triage accuracy overall, with a few exceptions (4/13, 31%). The main variables affecting the levels of diagnostic and triage accuracy were the severity and urgency of the condition, the use of artificial intelligence algorithms, and demographic questions. However, the impact of each variable differed across tools and studies, making it difficult to draw any solid conclusions. All included studies had at least one area with unclear risk of bias according to the revised Quality Assessment of Diagnostic Accuracy Studies-2 tool.
Conclusions: Although OSCs have potential to provide accessible and accurate health advice and triage recommendations to users, more research is needed to validate their triage and diagnostic accuracy before widescale adoption in community and health care settings. Future studies should aim to use a common methodology and agreed standard for evaluation to facilitate objective benchmarking and validation.
Trial registration: PROSPERO CRD42020215210; https://tinyurl.com/3949zw83.
Keywords: accuracy; diagnosis; digital triage; mobile phone; online symptom checker; safety; systematic review.
©Eva Riboli-Sasco, Austen El-Osta, Aos Alaa, Iman Webber, Manisha Karki, Marie Line El Asmar, Katie Purohit, Annabelle Painter, Benedict Hayhoe. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 02.06.2023.
Conflict of interest statement
Conflicts of Interest: AP previously worked for Babylon Health, an OSC provider (June 2019-July 2020). BH previously worked for Healthily, an OSC provider (Nov 2019-May 2021).
Figures
Similar articles
-
Digital and online symptom checkers and assessment services for urgent care to inform a new digital platform: a systematic review.Southampton (UK): NIHR Journals Library; 2019 Aug. Southampton (UK): NIHR Journals Library; 2019 Aug. PMID: 31433612 Free Books & Documents. Review.
-
Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas.Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article.
-
Digital and online symptom checkers and health assessment/triage services for urgent health problems: systematic review.BMJ Open. 2019 Aug 1;9(8):e027743. doi: 10.1136/bmjopen-2018-027743. BMJ Open. 2019. PMID: 31375610 Free PMC article.
-
Beyond the black stump: rapid reviews of health research issues affecting regional, rural and remote Australia.Med J Aust. 2020 Dec;213 Suppl 11:S3-S32.e1. doi: 10.5694/mja2.50881. Med J Aust. 2020. PMID: 33314144
-
The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review.NPJ Digit Med. 2022 Aug 17;5(1):118. doi: 10.1038/s41746-022-00667-w. NPJ Digit Med. 2022. PMID: 35977992 Free PMC article. Review.
Cited by
-
Importance of Patient History in Artificial Intelligence-Assisted Medical Diagnosis: Comparison Study.JMIR Med Educ. 2024 Apr 8;10:e52674. doi: 10.2196/52674. JMIR Med Educ. 2024. PMID: 38602313 Free PMC article.
-
A scoping review of triage approaches for the referral of patients with suspected inflammatory arthritis, from primary to rheumatology care.Rheumatol Int. 2024 Mar 26. doi: 10.1007/s00296-024-05575-8. Online ahead of print. Rheumatol Int. 2024. PMID: 38530455 Review.
-
[Short paths to diagnosis with artificial intelligence: systematic literature review on diagnostic decision support systems].Schmerz. 2024 Feb;38(1):19-27. doi: 10.1007/s00482-023-00777-8. Epub 2024 Jan 2. Schmerz. 2024. PMID: 38165492 German.
-
The Potential of ChatGPT as a Self-Diagnostic Tool in Common Orthopedic Diseases: Exploratory Study.J Med Internet Res. 2023 Sep 15;25:e47621. doi: 10.2196/47621. J Med Internet Res. 2023. PMID: 37713254 Free PMC article.
References
-
- Health workforce: the health workforce crisis. World Health Organization. 2009. Jun 24, [2023-05-04]. https://www.who.int/news-room/questions-and-answers/item/q-a-on-the-heal... .
-
- Alwashmi MF. The use of digital health in the detection and management of COVID-19. Int J Environ Res Public Health. 2020 Apr 23;17(8):906. doi: 10.3390/ijerph17082906. https://www.mdpi.com/resolver?pii=ijerph17082906 ijerph17082906 - DOI - PMC - PubMed
-
- Powley L, McIlroy G, Simons G, Raza K. Are online symptoms checkers useful for patients with inflammatory arthritis? BMC Musculoskelet Disord. 2016 Aug 24;17(1):362. doi: 10.1186/s12891-016-1189-2. https://bmcmusculoskeletdisord.biomedcentral.com/articles/10.1186/s12891... 10.1186/s12891-016-1189-2 - DOI - DOI - PMC - PubMed
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources
Medical