ArticleJournal of orthopaedics2024
The ability of artificial intelligence tools to formulate orthopaedic clinical decisions in comparison to human clinicians: An analysis of ChatGPT 3.5, ChatGPT 4, and Bard.
Article in Journal of orthopaedics, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 12 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
12 citing papers in PubMed, 1 synthesis or guideline pooled it, 25 citations in OpenAlex.
- Examining the Role of Large Language Models in Orthopedics: Systematic Review.Journal of medical Internet research · 2024Pooled it
- Artificial intelligence advancements for orthopaedic clinical reasoning: longitudinal assessment of newer models (ChatGPT-5, Grok-3, Gemini 2.5 Flash) compared to clinicians.Archives of orthopaedic and trauma surgery · 2026Article
- Evaluating deepresearch and deepthink in total knee arthroplasty patient education: ChatGPT-4o excels in comprehensiveness, Deepseek R1 leads in clarity and readability of orthopedic information.Joint diseases and related surgery · 2026Article
- ChatGPT models provide higher-quality but lower-readability responses than Google Gemini regarding anterior shoulder instability, with no added benefit of the orthopaedic expert plugin.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026Article
- Intelligent Rehabilitation: Advances in Artificial Intelligence for Musculoskeletal Rehabilitation: A Narrative Review.Orthopedic research and reviews · 2026Review
- Widely Available Large Language Models Are Not a Reliable Source to Address Medical Treatment Recommendations of Patients After a First-Time Anteroinferior Shoulder Dislocation.Arthroscopy, sports medicine, and rehabilitation · 2025Article
- Systematic Review on Large Language Models in Orthopaedic Surgery.Journal of clinical medicine · 2025Review
- Evaluating DeepResearch and DeepThink in anterior cruciate ligament surgery patient education: ChatGPT-4o excels in comprehensiveness, DeepSeek R1 leads in clarity and readability of orthopaedic information.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2025Article
- Evaluating the quality and readability of ChatGPT-generated patient-facing medical information in rhinology.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2025Article
- Assessing the Current Limitations of Large Language Models in Advancing Health Care Education.JMIR formative research · 2025Article
- Evaluating Large Language Model (LLM) Performance on Established Breast Classification Systems.Diagnostics (Basel, Switzerland) · 2024Article
- Exploring the Performance of ChatGPT in an Orthopaedic Setting and Its Potential Use as an Educational Tool.JB & JS open accessReview
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors at 2 institutions in 2 countries.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background: Recent advancements in artificial intelligence (AI) have sparked interest in its integration into clinical medicine and education. This study evaluates the performance of three AI tools compared to human clinicians in addressing complex orthopaedic decisions in real-world clinical cases. Questions/purposes: To evaluate the ability of commonly used AI tools to formulate orthopaedic clinical decisions in comparison to human clinicians. Patients and methods: The study used OrthoBullets Cases, a publicly available clinical cases collaboration platform where surgeons from around the world choose treatment options based on peer-reviewed standardised treatment polls. The clinical cases cover various orthopaedic categories. Three AI tools, (ChatGPT 3.5, ChatGPT 4, and Bard), were evaluated. Uniform prompts were used to input case information including questions relating to the case, and the AI tools' responses were analysed for alignment with the most popular response, within 10%, and within 20% of the most popular human responses. Results: In total, 8 clinical categories comprising of 97 questions were analysed. ChatGPT 4 demonstrated the highest proportion of most popular responses (proportion of most popular response: ChatGPT 4 68.0%, ChatGPT 3.5 40.2%, Bard 45.4%, Conclusions: While AI tools demonstrated potential use in educational contexts, their integration into clinical decision-making requires caution due to inconsistent responses and deviations from peer consensus. Future research should focus on specialised clinical AI tool development to maximise utility in clinical decision-making. Level of evidence: IV.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.