ArticleMayo Clinic proceedings. Digital health2023
Diagnostic Accuracy of Artificial Intelligence in Virtual Primary Care.
Article in Mayo Clinic proceedings. Digital health, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
17 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Opportunities, challenges, and requirements for Artificial Intelligence (AI) implementation in Primary Health Care (PHC): a systematic review.BMC primary care · 2025Pooled it
- Trial
- Alignment Between Cardiologists and AI-Driven Diagnostic Systems: Mixed Methods Study.Journal of medical Internet research · 2026Article
- Automated algorithm in catheter-associated urinary tract infection identification using digital health records: validity and barriers in implementation into clinical practice.BMC medical informatics and decision making · 2026Article
- Advancing conversational diagnostic AI with multimodal reasoning.Nature medicine · 2026Article
- Diagnostic performance of artificial intelligence for dermatological conditions: a systematic review focused on low- and middle-income countries to address resource constraints and improve access to specialist care.International journal of emergency medicine · 2025Review
- A Model Predicting Artificial Intelligence Use by Gastroenterology Nurses in Clinical Practice: A Cross-Sectional Multicenter Survey.Journal of gastroenterology and hepatology · 2025Article
- Beyond the Growth: A Registry-Based Analysis of Global Imbalances in Artificial Intelligence Clinical Trials.Healthcare (Basel, Switzerland) · 2025Article
- The relevance of clinical skills in an era of large language models: an adrenal insufficiency case study.International journal of medical education · 2025Article
- Recommendations for Clinicians, Technologists, and Healthcare Organizations on the Use of Generative Artificial Intelligence in Medicine: A Position Statement from the Society of General Internal Medicine.Journal of general internal medicine · 2025Article
- Clinical AI Scribes in primary care: accuracy, error severity and implications for clinical practice.BMJ digital health & AI · 2025Article
- Fully Automated Detection of the Appendix Using U-Net Deep Learning Architecture in CT Scans.Journal of clinical medicine · 2024Article
- Quality of interaction between clinicians and artificial intelligence systems. A systematic review.Future healthcare journal · 2024Review
- Generative artificial intelligence in neurology: Opportunities and risks.European journal of neurology · 2024Article
- A Transformative Future for Health Care: On the First Year of Mayo Clinic Proceedings: Digital Health.Mayo Clinic proceedings. Digital health · 2024Article
- Complexities and Questions Toward Artificial Intelligence for Diagnostic Support in Virtual Primary Care.Mayo Clinic proceedings. Digital health · 2023Article
- Needs assessment and prioritization of key medical specialties: A telemedicine roadmap based on a mixed-methods framework.Digital healthArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Objective: To evaluate the diagnostic accuracy of artificial intelligence (AI)-generated clinical diagnoses. Patients and Methods: A retrospective chart review of 102,059 virtual primary care clinical encounters from October 1, 2022, to January 31, 2023 was conducted. Patients underwent an AI medical interview, after which virtual care providers reviewed the interview summary and AI-provided differential diagnoses, communicated with patients, and finalized diagnoses and treatment plans. Our accuracy measures were agreement between AI diagnoses, virtual care providers, and blind adjudicators. We analyzed AI diagnostic agreement across different diagnoses, presenting symptoms, patient demographic characteristics such as race, and provider levels of experience. We also evaluated model performance improvement with retraining. Results: Providers selected an AI diagnosis in 84.2% (n = 85,976) of cases and the top-ranked AI diagnosis in 60.9% (n = 62,130) of cases. Agreement rates varied by diagnosis, with greater than or equal to 95% provider agreement with an AI diagnosis for 35 diagnoses (47% of cases, n = 47,679) and greater than or equal to 90% agreement for 57 diagnoses (69% of cases, n = 70,697). The average agreement rate for half of all presenting symptoms was greater than or equal to 90%. Adjusting for case mix, diagnostic accuracy exhibited minimal variation across demographic characteristics. The adjudicators' consensus diagnosis, reached in 58.2% (n = 128) of adjudicated cases was always included in the AI differential diagnosis. Provider experience did not affect agreement, and model retraining increased diagnostic accuracy for retrained conditions from 96.6% to 98.0%. Conclusion: Our findings show that agreement between AI and provider diagnoses is high in most cases in the setting of this study. The results highlight the potential for AI to enhance primary care disease diagnosis and patient triage, with the capacity to improve over time.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.