ArticleJournal of clinical medicine2023
Are ChatGPT's Free-Text Responses on Periprosthetic Joint Infections of the Hip and Knee Reliable and Useful?
Article in Journal of clinical medicine, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
17 citing papers in PubMed, 1 synthesis or guideline pooled it, 31 citations in OpenAlex.
- Examining the Role of Large Language Models in Orthopedics: Systematic Review.Journal of medical Internet research · 2024Pooled it
- The Reliability of Human Evaluation of Large Language Models in Health Care Settings: Scoping Review.Journal of medical Internet research · 2026Article
- The Application of artificial intelligence in periprosthetic joint infection.Journal of advanced research · 2026Review
- AI-Generated Antibiotic Therapies for Acute Periprosthetic Joint Infections with Implant Retention in Comparison with an Interdisciplinary Team.Antibiotics (Basel, Switzerland) · 2025Article
- SMART 2.0 Statistical Metabolomics Analysis: An R Tool 2.0.Analytical chemistry · 2025Article
- Exploring Patient Perspectives, Engagement, and Output Quality in Doctor-Supervised Use of Artificial Intelligence During Informed Consent Consultation With ChatGPT and Retrieval Augmented Generation (RAG): Quantitative Exploratory Study.Journal of medical Internet research · 2025Article
- Beyond Cultures: The Evolving Role of Molecular Diagnostics, Synovial Biomarkers and Artificial Intelligence in the Diagnosis of Prosthetic Joint Infections.Journal of clinical medicine · 2025Review
- Battle of the Bots: Solving Clinical Cases in Osteoarticular Infections With Large Language Models.Mayo Clinic proceedings. Digital health · 2025Article
- Evaluating Large Language Models for Preoperative Patient Education in Superior Capsular Reconstruction: Comparative Study of Claude, GPT, and Gemini.JMIR perioperative medicine · 2025Article
- Current applications and challenges in large language models for patient care: a systematic review.Communications medicine · 2025Article
- Can we rely on artificial intelligence to guide antimicrobial therapy? A systematic literature review.Antimicrobial stewardship & healthcare epidemiology : ASHE · 2025Article
- Assessing the quality of ChatGPT's responses to questions related to radiofrequency ablation for varicose veins.Journal of vascular surgery. Venous and lymphatic disorders · 2025Article
- GPT-based chatbot tools are still unreliable in the management of prosthetic joint infections.Musculoskeletal surgery · 2024Article
- Effectiveness of a large language model for clinical information retrieval regarding shoulder arthroplasty.Journal of experimental orthopaedics · 2024Article
- A framework for human evaluation of large language models in healthcare derived from literature review.NPJ digital medicine · 2024Article
- ChatGPT May Offer an Adequate Substitute for Informed Consent to Patients Prior to Total Knee Arthroplasty-Yet Caution Is Needed.Journal of personalized medicine · 2024Article
- ChatGPT-4 Knows Its A B C D E but Cannot Cite Its Source.JB & JS open accessReview
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors at 1 institution in 1 country.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundThis study aimed to evaluate ChatGPT's performance on questions about periprosthetic joint infections (PJI) of the hip and knee.
methodsTwenty-seven questions from the 2018 International Consensus Meeting on Musculoskeletal Infection were selected for response generation. The free-text responses were evaluated by three orthopedic surgeons using a five-point Likert scale. Inter-rater reliability (IRR) was assessed via Fleiss' kappa (FK).
resultsOverall, near-perfect IRR was found for disagreement on the presence of factual errors (FK: 0.880, 95% CI [0.724, 1.035],
conclusionsChatGPT's free-text responses to complex orthopedic questions were predominantly reliable and useful for orthopedic surgeons and patients. Given variations in performance by question and subtopic, consulting additional sources and exercising careful interpretation should be emphasized for reliable medical decision-making.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.