SynthesisBMJ open2019
Digital and online symptom checkers and health assessment/triage services for urgent health problems: systematic review.
Synthesis in BMJ open, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 112 papers, 10 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
112 citing papers in PubMed, 10 syntheses or guidelines pooled it.
- Safety and Efficacy of Digital Check-in and Triage Kiosks in Emergency Departments: Systematic Review.Journal of medical Internet research · 2025Pooled it
- The Role of Language in Remote Healthcare Triage: A Meta-Aggregative Review.Journal of advanced nursing · 2025Pooled it
- Artificial intelligence-based risk assessment tools for sexual, reproductive and mental health: a systematic review.BMC medical informatics and decision making · 2025Pooled it
- Impact of the clinically oriented roles of a general practice receptionist: a systematic review with narrative synthesis.The British journal of general practice : the journal of the Royal College of General Practitioners · 2025Pooled it
- Pooled it
- Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review.Journal of medical Internet research · 2023Pooled it
- Understanding How the Design and Implementation of Online Consultations Affect Primary Care Quality: Systematic Review of Evidence With Recommendations for Designers, Providers, and Researchers.Journal of medical Internet research · 2022Pooled it
- Challenges in evaluating the accuracy of AI-containing digital triage systems: A systematic review.PloS one · 2022Pooled it
- Artificial Intelligence Techniques That May Be Applied to Primary Care Data to Facilitate Earlier Diagnosis of Cancer: Systematic Review.Journal of medical Internet research · 2021Pooled it
- Proposal of a novel Artificial Intelligence Distribution Service platform for healthcare.F1000Research · 2021Pooled it
- Can clinical decision support systems be an asset in medical education? An experimental approach.BMC medical education · 2023Trial
- Trial
- Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial.Arthritis research & therapy · 2021Trial
- Telephone triage in urgent unscheduled primary care in 16 European countries: a cross-national questionnaire-based expert study.Scandinavian journal of primary health care · 2026Article
- Telehealth and access to acute primary care in Europe: Balancing opportunity and equity.The European journal of general practice · 2026Article
- [Use of a digital health navigator in the emergency department: experiences and acceptance among patients and physicians].Bundesgesundheitsblatt, Gesundheitsforschung, Gesundheitsschutz · 2026Article
- Authentication status and AI triage concordance among care seekers in a US health system.npj health systems · 2026Article
- A light-weight symptom checker and its methodological validation.Veterinary research communications · 2026Article
- Real-World Evaluation of the Eye+Dot Online Triage Support Tool in Community Optometry Practices: Mixed Methods Evaluation Study.JMIR human factors · 2026Article
- AI Triage in Primary Care: Building Safer and More Equitable Real-World Evidence.Journal of medical Internet research · 2026Article
52 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
Abstract
objectivesIn England, the NHS111 service provides assessment and triage by telephone for urgent health problems. A digital version of this service has recently been introduced. We aimed to systematically review the evidence on digital and online symptom checkers and similar services.
designSystematic review. DATA SOURCES: We searched Medline, Embase, the Cochrane Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Health Management Information Consortium, Web of Science and ACM Digital Library up to April 2018, supplemented by phrase searches for known symptom checkers and citation searching of key studies. ELIGIBILITY CRITERIA: Studies of any design that evaluated a digital or online symptom checker or health assessment service for people seeking advice about an urgent health problem. DATA EXTRACTION AND SYNTHESIS: Data extraction and quality assessment (using the Cochrane Collaboration version of QUADAS for diagnostic accuracy studies and the National Heart, Lung and Blood Institute tool for observational studies) were done by one reviewer with a sample checked for accuracy and consistency. We performed a narrative synthesis of the included studies structured around pre-defined research questions and key outcomes.
resultsWe included 29 publications (27 studies). Evidence on patient safety was weak. Diagnostic accuracy varied between different systems but was generally low. Algorithm-based triage tended to be more risk averse than that of health professionals. There was very limited evidence on patients' compliance with online triage advice. Study participants generally expressed high levels of satisfaction, although in mainly uncontrolled studies. Younger and more highly educated people were more likely to use these services.
conclusionsThe English 'digital 111' service has been implemented against a background of uncertainty around the likely impact on important outcomes. The health system may need to respond to short-term changes and/or shifts in demand. The popularity of online and digital services with younger and more educated people has implications for health equity. PROSPERO REGISTRATION NUMBER: CRD42018093564.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.