ArticleClinics in orthopedic surgery2024
Performance of ChatGPT on Solving Orthopedic Board-Style Questions: A Comparative Analysis of ChatGPT 3.5 and ChatGPT 4.
Article in Clinics in orthopedic surgery, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 12 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
12 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Artificial intelligence in orthopaedic education, training and research: a systematic review.BMC medical education · 2025Pooled it
- Accuracy of Large Language Models When Answering Clinical Research Questions: Systematic Review and Network Meta-Analysis.Journal of medical Internet research · 2025Pooled it
- Flatfoot and artificial intelligence: Can language models be trusted for patient education?Medicine · 2026Article
- Can ChatGPT pass the polish national medical specialization examination in orthopedics and traumatology?Archives of orthopaedic and trauma surgery · 2026Article
- Benchmarking Open-Source Vision Language Models in Orthopedic In-Training Examination: A Comparison with Residents, Domain-Specific Evaluation, and Parameter Scaling.Clinics in orthopedic surgery · 2026Article
- Reasoning-optimised large language models reach near-expert accuracy on board-style orthopaedic exams: A multi-model comparison on 702 multiple-choice questions.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026Article
- Large language models in orthopedics: An exploratory research trend analysis and machine learning classification.Journal of orthopaedics · 2025Article
- Exploring ChatGPT's Efficacy in Orthopaedic Arthroplasty Questions Compared to Adult Reconstruction Surgeons.Arthroplasty today · 2025Article
- Is it a pediatric orthopaedic urgency or not? Can ChatGPT answer this question?Journal of orthopaedic surgery and research · 2025Article
- Evaluating the Effectiveness of Large Language Models in Providing Patient Education for Chinese Patients With Ocular Myasthenia Gravis: Mixed Methods Study.Journal of medical Internet research · 2025Article
- Review
- Opportunities and Challenges of Chatbots in Ophthalmology: A Narrative Review.Journal of personalized medicine · 2024Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background: The application of artificial intelligence and large language models in the medical field requires an evaluation of their accuracy in providing medical information. This study aimed to assess the performance of Chat Generative Pre-trained Transformer (ChatGPT) models 3.5 and 4 in solving orthopedic board-style questions. Methods: A total of 160 text-only questions from the Orthopedic Surgery Department at Seoul National University Hospital, conforming to the format of the Korean Orthopedic Association board certification examinations, were input into the ChatGPT 3.5 and ChatGPT 4 programs. The questions were divided into 11 subcategories. The accuracy rates of the initial answers provided by Chat GPT 3.5 and ChatGPT 4 were analyzed. In addition, inconsistency rates of answers were evaluated by regenerating the responses. Results: ChatGPT 3.5 answered 37.5% of the questions correctly, while ChatGPT 4 showed an accuracy rate of 60.0% ( Conclusions: ChatGPT 4 showed the ability to pass orthopedic board-style examinations, outperforming ChatGPT 3.5 in accuracy rate. However, inconsistencies in response generation and instances of incorrect answers with misleading explanations require caution when applying ChatGPT in clinical settings or for educational purposes.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.