ArticleRheumatology international2024
Diagnostic accuracy of a large language model in rheumatology: comparison of physician and ChatGPT-4.
Article in Rheumatology international, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 69 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
69 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Accuracy of Large Language Models When Answering Clinical Research Questions: Systematic Review and Network Meta-Analysis.Journal of medical Internet research · 2025Pooled it
- Large Language Models for Chatbot Health Advice Studies: A Systematic Review.JAMA network open · 2025Pooled it
- Large language models enhance diagnostic reasoning of medical students in rheumatology: a randomized controlled trial.BMC medical education · 2026Trial
- RhePort 1.3 enhances early identification of inflammatory rheumatic diseases: a prospective study in German rheumatology settings.Rheumatology international · 2025Trial
- Comparison of Two Symptom Checkers (Ada and Symptoma) in the Emergency Department: Randomized, Crossover, Head-to-Head, Double-Blinded Study.Journal of medical Internet research · 2024Trial
- Diagnostic Accuracy of a Mobile AI-Based Symptom Checker and a Web-Based Self-Referral Tool in Rheumatology: Multicenter Randomized Controlled Trial.Journal of medical Internet research · 2024Trial
- Automating the triage of rheumatology outpatient referrals: a comparative evaluation of 23 large language models under simple and advanced prompting.Rheumatology international · 2026Article
- Large language models in the diagnosis and treatment of immunosuppression-related infections in rheumatic diseases: a comparative, vignette-based study.Rheumatology international · 2026Article
- Leveraging time-series electronic health records with large language models for chronic kidney disease diagnosis in primary care.BMJ health & care informatics · 2026Article
- Artificial Intelligence and Machine Learning in Rheumatology and Systemic Inflammatory Diseases: From Pattern Recognition to Signal Analysis and Clinical Decision Support.Journal of clinical medicine · 2026Review
- Structured prompting as reusable clinical tools.EULAR rheumatology open · 2026Article
- Stepwise Diagnostic Evaluation of Chinese Large Language Models: Comparative Study of Common and Rare Diseases.Journal of medical Internet research · 2026Article
- Moderate-to-substantial agreement of ChatGPT-5 for Kellgren-Lawrence grading on synthetic knee radiographs: a controlled cross-sectional observer agreement study.Rheumatology international · 2026Article
- Generating Question Prompt Lists From Electronic Health Record Data Using Large Language Models: Iterative Evaluation Study.Journal of medical Internet research · 2026Article
- Post-stroke rehabilitation in inflammatory rheumatic diseases: outcome measures, digital tools, and patient education perspectives.Rheumatology international · 2026Review
- [Survey on primary care for rheumatology patients in Lower Franconia and the contribution of digital health].Zeitschrift fur Rheumatologie · 2026Article
- Bridging the Precision Gap in Rheumatoid Arthritis: Spatial Transcriptomics, Spatial Proteomics, and Artificial Intelligence in Precision Health.Biomedicines · 2026Review
- Artificial Intelligence (AI) in rheumatology: a comparative evaluation of the ChatGPT and DeepSeek application.BMC rheumatology · 2026Article
- Independent and collaborative performance of large language models and healthcare professionals in diagnosis and triage.NPJ digital medicine · 2026Article
- Evaluation of large Language model performance on Persian rheumatology board exams: accuracy and clinical reasoning of GPT-4o vs. GPT-5.1.Scientific reports · 2026Article
9 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Pre-clinical studies suggest that large language models (i.e., ChatGPT) could be used in the diagnostic process to distinguish inflammatory rheumatic (IRD) from other diseases. We therefore aimed to assess the diagnostic accuracy of ChatGPT-4 in comparison to rheumatologists. For the analysis, the data set of Gräf et al. (2022) was used. Previous patient assessments were analyzed using ChatGPT-4 and compared to rheumatologists' assessments. ChatGPT-4 listed the correct diagnosis comparable often to rheumatologists as the top diagnosis 35% vs 39% (p = 0.30); as well as among the top 3 diagnoses, 60% vs 55%, (p = 0.38). In IRD-positive cases, ChatGPT-4 provided the top diagnosis in 71% vs 62% in the rheumatologists' analysis. Correct diagnosis was among the top 3 in 86% (ChatGPT-4) vs 74% (rheumatologists). In non-IRD cases, ChatGPT-4 provided the correct top diagnosis in 15% vs 27% in the rheumatologists' analysis. Correct diagnosis was among the top 3 in non-IRD cases in 46% of the ChatGPT-4 group vs 45% in the rheumatologists group. If only the first suggestion for diagnosis was considered, ChatGPT-4 correctly classified 58% of cases as IRD compared to 56% of the rheumatologists (p = 0.52). ChatGPT-4 showed a slightly higher accuracy for the top 3 overall diagnoses compared to rheumatologist's assessment. ChatGPT-4 was able to provide the correct differential diagnosis in a relevant number of cases and achieved better sensitivity to detect IRDs than rheumatologist, at the cost of lower specificity. The pilot results highlight the potential of this new technology as a triage tool for the diagnosis of IRD.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.