ArticleJournal of medical Internet research2022
Online Symptom Checkers: Recommendations for a Vignette-Based Clinical Evaluation Standard.
Article in Journal of medical Internet research, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
19 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review.Journal of medical Internet research · 2023Pooled it
- Evaluating digital triage symptom checker with historical triage-related adverse events.Scandinavian journal of primary health care · 2026Article
- A light-weight symptom checker and its methodological validation.Veterinary research communications · 2026Article
- Increasing Large Language Model Accuracy for Care-Seeking Advice Using Prompts Reflecting Human Reasoning Strategies in the Real World: Validation Study.JMIR biomedical engineering · 2026Article
- How to Evaluate the Accuracy of Symptom Checkers and Diagnostic Decision Support Systems: Symptom Checker Accuracy Reporting Framework (SCARF).JMIR human factors · 2026Article
- Factors Influencing the Use of Online Symptom Checkers in the United Kingdom: Cross-Sectional Study.JMIR formative research · 2025Article
- Article
- Accuracy of online symptom assessment applications, large language models, and laypeople for self-triage decisions.NPJ digital medicine · 2025Article
- Technology-supported self-triage decision making.npj health systems · 2025Article
- The RepVig framework for designing use-case specific representative vignettes and evaluating triage accuracy of laypeople and symptom assessment applications.Scientific reports · 2024Article
- Triage Accuracy and the Safety of User-Initiated Symptom Assessment With an Electronic Symptom Checker in a Real-Life Setting: Instrument Validation Study.JMIR human factors · 2024Article
- Evaluating ChatGPT-4's Accuracy in Identifying Final Diagnoses Within Differential Diagnoses Compared With Those of Physicians: Experimental Study for Diagnostic Cases.JMIR formative research · 2024Article
- Controlling Inputter Variability in Vignette Studies Assessing Web-Based Symptom Checkers: Evaluation of Current Practice and Recommendations for Isolated Accuracy Metrics.JMIR formative research · 2024Article
- Longitudinal Changes in Diagnostic Accuracy of a Differential Diagnosis List Developed by an AI-Based Symptom Checker: Retrospective Observational Study.JMIR formative research · 2024Article
- Article
- Evaluating ChatGPT-4's Diagnostic Accuracy: Impact of Visual Data Integration.JMIR medical informatics · 2024Article
- Statistical refinement of patient-centered case vignettes for digital health research.Frontiers in digital health · 2024Article
- Assessment of a Digital Symptom Checker Tool's Accuracy in Suggesting Reproductive Health Conditions: Clinical Vignettes Study.JMIR mHealth and uHealth · 2023Article
- Diagnostic performance of generative artificial intelligences for a series of complex case reports.Digital healthArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
The use of patient-facing online symptom checkers (OSCs) has expanded in recent years, but their accuracy, safety, and impact on patient behaviors and health care systems remain unclear. The lack of a standardized process of clinical evaluation has resulted in significant variation in approaches to OSC validation and evaluation. The aim of this paper is to characterize a set of congruent requirements for a standardized vignette-based clinical evaluation process of OSCs. Discrepancies in the findings of comparative studies to date suggest that different steps in OSC evaluation methodology can significantly influence outcomes. A standardized process with a clear specification for vignette-based clinical evaluation is urgently needed to guide developers and facilitate the objective comparison of OSCs. We propose 15 recommendation requirements for an OSC evaluation standard. A third-party evaluation process and protocols for prospective real-world evidence studies should also be prioritized to quality assure OSC assessment.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.