ReviewNPJ digital medicine2022
The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review.
Review in NPJ digital medicine, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 88 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
88 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- AI applications in veterinary digital health: a systematic survey.Frontiers in veterinary science · 2026Pooled it
- Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review.Journal of medical Internet research · 2023Pooled it
- Trial
- Impact of a Symptom Checker App on Patient-Physician Interaction Among Self-Referred Walk-In Patients in the Emergency Department: Multicenter, Parallel-Group, Randomized, Controlled Trial.Journal of medical Internet research · 2025Trial
- Comparison of Two Symptom Checkers (Ada and Symptoma) in the Emergency Department: Randomized, Crossover, Head-to-Head, Double-Blinded Study.Journal of medical Internet research · 2024Trial
- Evaluating digital triage symptom checker with historical triage-related adverse events.Scandinavian journal of primary health care · 2026Article
- Telehealth and access to acute primary care in Europe: Balancing opportunity and equity.The European journal of general practice · 2026Article
- Patient safety practices in an innovative digital landscape: an interview study on triage in Swedish digital primary care.Scandinavian journal of primary health care · 2026Article
- Authentication status and AI triage concordance among care seekers in a US health system.npj health systems · 2026Article
- Evaluation of the Community Urgent Eyecare Service in Greater Manchester, Including the Impact of Independent Prescribing.Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians (Optometrists) · 2026Article
- Agreement and Classification Performance of a Mobile Self-Triage Application Based on the Manchester Triage System in Hemodynamically Stable Adult Walk-In Emergency Department Patients: A Prospective Observational Study.Journal of clinical medicine · 2026Article
- Patient-Facing AI System for Symptom Guidance Using Simulated Encounters and Physician Review: Analytical Validation Study.JMIR formative research · 2026Article
- AI-Based Diagnostic Platform Capabilities With Lyme Disease as a Use Case: Integrative Exploration.Online journal of public health informatics · 2026Article
- Article
- Ethical Considerations in Personal Health Large Language Models.Journal of medical Internet research · 2026Article
- A light-weight symptom checker and its methodological validation.Veterinary research communications · 2026Article
- Appropriateness and utility of a clinical decision support system at the digital front door.NPJ digital medicine · 2026Article
- Artificial intelligence across the cardiovascular diagnostic pathway: a case-based narrative review.Cardiovascular diagnosis and therapy · 2026Review
- Seven deadly sins in artificial intelligence for digital medicine.NPJ digital medicine · 2026Article
- Real-World Performance of a New Online Eye Symptom Triage Tool (Eye+Dot) in an Emergency Eye Clinic: Mixed Methods Evaluation Study.JMIR human factors · 2026Article
28 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Digital and online symptom checkers are an increasingly adopted class of health technologies that enable patients to input their symptoms and biodata to produce a set of likely diagnoses and associated triage advice. However, concerns regarding the accuracy and safety of these symptom checkers have been raised. This systematic review evaluates the accuracy of symptom checkers in providing diagnoses and appropriate triage advice. MEDLINE and Web of Science were searched for studies that used either real or simulated patients to evaluate online or digital symptom checkers. The primary outcomes were the diagnostic and triage accuracy of the symptom checkers. The QUADAS-2 tool was used to assess study quality. Of the 177 studies retrieved, 10 studies met the inclusion criteria. Researchers evaluated the accuracy of symptom checkers using a variety of medical conditions, including ophthalmological conditions, inflammatory arthritides and HIV. A total of 50% of the studies recruited real patients, while the remainder used simulated cases. The diagnostic accuracy of the primary diagnosis was low across included studies (range: 19-37.9%) and varied between individual symptom checkers, despite consistent symptom data input. Triage accuracy (range: 48.8-90.1%) was typically higher than diagnostic accuracy. Overall, the diagnostic and triage accuracy of symptom checkers are variable and of low accuracy. Given the increasing push towards adopting this class of technologies across numerous health systems, this study demonstrates that reliance upon symptom checkers could pose significant patient safety hazards. Large-scale primary studies, based upon real-world data, are warranted to demonstrate the adequate performance of these technologies in a manner that is non-inferior to current best practices. Moreover, an urgent assessment of how these systems are regulated and implemented is required.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.