ArticleJournal of medical Internet research2022
Triage Accuracy of Symptom Checker Apps: 5-Year Follow-up Evaluation.
Article in Journal of medical Internet research, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 55 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
55 citing papers in PubMed.
- Impact of a Symptom Checker App on Patient-Physician Interaction Among Self-Referred Walk-In Patients in the Emergency Department: Multicenter, Parallel-Group, Randomized, Controlled Trial.Journal of medical Internet research · 2025Trial
- Can clinical decision support systems be an asset in medical education? An experimental approach.BMC medical education · 2023Trial
- Evaluating digital triage symptom checker with historical triage-related adverse events.Scandinavian journal of primary health care · 2026Article
- [Use of a digital health navigator in the emergency department: experiences and acceptance among patients and physicians].Bundesgesundheitsblatt, Gesundheitsforschung, Gesundheitsschutz · 2026Article
- Concordance between an artificial intelligence self-triage programme and physical triage.Emergency medicine journal : EMJ · 2026Observational
- Patient-Facing AI System for Symptom Guidance Using Simulated Encounters and Physician Review: Analytical Validation Study.JMIR formative research · 2026Article
- Increasing Large Language Model Accuracy for Care-Seeking Advice Using Prompts Reflecting Human Reasoning Strategies in the Real World: Validation Study.JMIR biomedical engineering · 2026Article
- Acceptable accuracy for medical AI: a survey of physicians and the general population in Sweden.BMJ health & care informatics · 2026Article
- Safety-First Framework for AI-Enabled Anamnesis in Head and Neck Surgery: Evidence Synthesis from a Narrative Review.Journal of clinical medicine · 2026Review
- Article
- Evaluating the accuracy of ChatGPT model versions for giving care-seeking advice.Communications medicine · 2026Article
- Independent and collaborative performance of large language models and healthcare professionals in diagnosis and triage.NPJ digital medicine · 2026Article
- How to Evaluate the Accuracy of Symptom Checkers and Diagnostic Decision Support Systems: Symptom Checker Accuracy Reporting Framework (SCARF).JMIR human factors · 2026Article
- Article
- Accuracy of online symptom assessment applications, large language models, and laypeople for self-triage decisions.NPJ digital medicine · 2025Article
- Exploring Laypersons' Experiences With a Mobile Symptom Checker App as an Interface Between eHealth Literacy, Health Literacy, and Health-Related Behavior: Qualitative Interview Study.JMIR formative research · 2025Article
- Preliminary analysis of the impact of lab results on large language model generated differential diagnoses.NPJ digital medicine · 2025Article
- Technology-supported self-triage decision making.npj health systems · 2025Article
- Use of artificial intelligence for reverse referral between a hospital emergency department and a primary urgent care center.Frontiers in digital health · 2025Article
- The RepVig framework for designing use-case specific representative vignettes and evaluating triage accuracy of laypeople and symptom assessment applications.Scientific reports · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundSymptom checkers are digital tools assisting laypersons in self-assessing the urgency and potential causes of their medical complaints. They are widely used but face concerns from both patients and health care professionals, especially regarding their accuracy. A 2015 landmark study substantiated these concerns using case vignettes to demonstrate that symptom checkers commonly err in their triage assessment.
objectiveThis study aims to revisit the landmark index study to investigate whether and how symptom checkers' capabilities have evolved since 2015 and how they currently compare with laypersons' stand-alone triage appraisal.
methodsIn early 2020, we searched for smartphone and web-based applications providing triage advice. We evaluated these apps on the same 45 case vignettes as the index study. Using descriptive statistics, we compared our findings with those of the index study and with publicly available data on laypersons' triage capability.
resultsWe retrieved 22 symptom checkers providing triage advice. The median triage accuracy in 2020 (55.8%, IQR 15.1%) was close to that in 2015 (59.1%, IQR 15.5%). The apps in 2020 were less risk averse (odds 1.11:1, the ratio of overtriage errors to undertriage errors) than those in 2015 (odds 2.82:1), missing >40% of emergencies. Few apps outperformed laypersons in either deciding whether emergency care was required or whether self-care was sufficient. No apps outperformed the laypersons on both decisions.
conclusionsTriage performance of symptom checkers has, on average, not improved over the course of 5 years. It decreased in 2 use cases (advice on when emergency care is required and when no health care is needed for the moment). However, triage capability varies widely within the sample of symptom checkers. Whether it is beneficial to seek advice from symptom checkers depends on the app chosen and on the specific question to be answered. Future research should develop resources (eg, case vignette repositories) to audit the capabilities of symptom checkers continuously and independently and provide guidance on when and to whom they should be recommended.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.