ArticleJoint diseases and related surgery2025
Evaluation of ChatGPT-4o's answers to questions about hip arthroscopy from the patient perspective.
Article in Joint diseases and related surgery, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
17 citing papers in PubMed, 1 synthesis or guideline pooled it.
- ChatGPT provides accurate and safe responses to patient questions on hip arthroscopy, while completeness remains variable: A systematic review and single-arm meta-analysis.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026Pooled it
- The Reliability of Human Evaluation of Large Language Models in Health Care Settings: Scoping Review.Journal of medical Internet research · 2026Article
- Evaluating deepresearch and deepthink in total knee arthroplasty patient education: ChatGPT-4o excels in comprehensiveness, Deepseek R1 leads in clarity and readability of orthopedic information.Joint diseases and related surgery · 2026Article
- ChatGPT delivers mostly satisfactory but occasionally inaccurate and potentially unsafe answers to hip arthroscopy questions during the surgeon's learning curve.Journal of experimental orthopaedics · 2026Article
- Assessing The Ability of ChatGPT 5.2 to Answer Patient Questions Regarding Hip Avascular Necrosis.Acta orthopaedica et traumatologica turcica · 2026Article
- Gemini 1.5 Flash provides the most reliable content while ChatGPT-4o offers the highest readability for patient education on meniscal tears.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026Article
- Evaluating ChatGPT-5's Performance in Answering Common Patient Questions About Femoroacetabular Impingement and Hip Arthroscopy.Indian journal of orthopaedics · 2026Article
- Benchmarking Open-Source Vision Language Models in Orthopedic In-Training Examination: A Comparison with Residents, Domain-Specific Evaluation, and Parameter Scaling.Clinics in orthopedic surgery · 2026Article
- ChatGPT models provide higher-quality but lower-readability responses than Google Gemini regarding anterior shoulder instability, with no added benefit of the orthopaedic expert plugin.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026Article
- Evaluation of ChatGPT-5.2 responses to frequently asked questions about benign bone tumors.Frontiers in oncology · 2026Article
- Assessment of large language models in musculoskeletal radiological anatomy: A comparative study with radiologists.Joint diseases and related surgery · 2026Article
- Evaluating the accuracy and patient perception of AI-generated answers on pectus surgery.Journal of cardiothoracic surgery · 2025Article
- Assessing the Quality and Accuracy of ChatGPT-3.5 Responses to Patient Questions About Hip Arthroscopy.Medicina (Kaunas, Lithuania) · 2025Observational
- Evaluating ChatGPT's Utility in Biologic Therapy for Systemic Lupus Erythematosus: Comparative Study of ChatGPT and Google Web Search.JMIR formative research · 2025Article
- The assessment of ChatGPT-4's performance compared to expert's consensus on chronic lateral ankle instability.Journal of experimental orthopaedics · 2025Article
- Can ChatGPT pass the Turkish Orthopedics and Traumatology Board Examination? Turkish orthopedic surgeons versus artificial intelligence.Ulusal travma ve acil cerrahi dergisi = Turkish journal of trauma & emergency surgery : TJTES · 2025Observational
- Artificial intelligence in medical and biological research: promise and perils of ChatGPT and DeepSeek in advancing healthcare.Turkish journal of biology = Turk biyoloji dergisi · 2025Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectivesThis study aimed to evaluate the responses provided by ChatGPT-4o to the most frequently asked questions by patients regarding hip arthroscopy. MATERIALS AND
methodsIn this cross-sectional survey study, a new Google account without a search history was created to determine the 20 most frequently asked questions about hip arthroscopy via Google. These questions were asked to a new ChatGPT-4o account on June 1, 2024, and the responses were recorded. Ten orthopedic surgeons specializing in sports surgery rated the responses using a rating scale to assess relevance, accuracy, clarity, and completeness. The responses were scored on a scale from 1 to 5, with 1 being the worst and 5 being the best. The interrater reliability assessed via the intraclass correlation coefficient (ICC).
resultsThe lowest score given by the surgeons for any response was 4/5 in each subcategory. The highest mean scores were in accuracy and clarity, followed by relevance, with completeness receiving the lowest scores. The overall mean score was 4.49±0.16. Interrater reliability showed insufficient overall agreement (ICC=0.004, p=0.383), with the highest agreement in clarity (ICC=0.039, p=0.131) and the lowest in accuracy (ICC=-0.019, p=0.688).
conclusionThe study confirms our hypothesis that ChatGPT-4o provides above-average quality responses to frequently asked questions about hip arthroscopy, as evidenced by the high scores in relevance, accuracy, clarity, and completeness. However, it is still advisable to consult orthopedic specialists on the subject, incorporating ChatGPT's suggestions during the final decision-making process.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.