ArticleFrontiers in medicine2024
Evaluation of large language models as a diagnostic aid for complex medical cases.
Article in Frontiers in medicine, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 27 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
27 citing papers in PubMed, 1 synthesis or guideline pooled it.
- A systematic review of large language model (LLM) evaluations in clinical medicine.BMC medical informatics and decision making · 2025Pooled it
- Systematic benchmarking demonstrates large language models have not reached the diagnostic accuracy of traditional rare-disease decision support tools.European journal of human genetics : EJHG · 2026Article
- Grounding large language models in clinical diagnostics.Nature communications · 2026Article
- Affiliation in human-AI interactions is based on shared psychological traits.Communications psychology · 2026Article
- Accuracy, quality, and readability analyses of responses from large language models to questions on pediatric dental sedation.BMC oral health · 2026Article
- Advancing Medical Safety in AI Systems: Reflections on Reverse Prompting for Subtle Misinformation Detection.Journal of medical systems · 2026Article
- Artificial Intelligence's Capacity to Detect Subtle Medical Misinformation: A Novel Reverse Prompting Approach.Journal of medical systems · 2025Article
- AI-enabled language models (LMs) to large language models (LLMs) and multimodal large language models (MLLMs) in drug discovery and development.Journal of advanced research · 2025Review
- Performance of Large Language Models in Diagnosing Rare Hematologic Diseases and the Impact of Their Diagnostic Outputs on Physicians: Combined Retrospective and Prospective Study.Journal of medical Internet research · 2025Article
- A Multimodal Large Language Model as an End-to-End Classifier of Thyroid Nodule Malignancy Risk: Usability Study.JMIR formative research · 2025Article
- Application of Large Language Models in Stroke Rehabilitation Health Education: 2-Phase Study.Journal of medical Internet research · 2025Article
- Implementing Large Language Models in Health Care: Clinician-Focused Review With Interactive Guideline.Journal of medical Internet research · 2025Review
- Large language models and their performance for the diagnosis of histoplasmosis.PLoS neglected tropical diseases · 2025Article
- Article
- Large Language Models in Medical Diagnostics: Scoping Review With Bibliometric Analysis.Journal of medical Internet research · 2025Article
- Comparative analysis of large language models in clinical diagnosis: performance evaluation across common and complex medical cases.JAMIA open · 2025Article
- Medical accuracy of artificial intelligence chatbots in oncology: a scoping review.The oncologist · 2025Article
- A systematic review and meta-analysis of diagnostic performance comparison between generative AI and physicians.NPJ digital medicine · 2025Article
- Enhancing diagnostic capability with multi-agents conversational large language models.NPJ digital medicine · 2025Article
- Urinary Bladder Acute Inflammations and Nephritis of the Renal Pelvis: Diagnosis Using Fine-Tuned Large Language Models.Journal of personalized medicine · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background: The use of large language models (LLM) has recently gained popularity in diverse areas, including answering questions posted by patients as well as medical professionals. Objective: To evaluate the performance and limitations of LLMs in providing the correct diagnosis for a complex clinical case. Design: Seventy-five consecutive clinical cases were selected from the Massachusetts General Hospital Case Records, and differential diagnoses were generated by OpenAI's GPT3.5 and 4 models. Results: The mean number of diagnoses provided by the Massachusetts General Hospital case discussants was 16.77, by GPT3.5 30 and by GPT4 15.45 ( Conclusions and relevance: The GPT4 model was able to generate a differential diagnosis list with the correct diagnosis in approximately two thirds of cases, but the most likely diagnosis was often incorrect for both models. In its current state, this tool can at most be used as an aid to expand on potential diagnostic considerations for a case, and future LLMs should be trained which account for the discrepancy between disease incidence and availability in the literature.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.