ArticleBrain & spine2024
Are large language models valid tools for patient information on lumbar disc herniation? The spine surgeons' perspective.
Article in Brain & spine, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed.
- Alignment Between Large Language Models and Consensus in Endoscopic Spinal Disc Surgery: A Comparative Analysis Against the Neurocore-SENSED.JOR spine · 2026Article
- Large Language Models in Spine Surgery : A Narrative Review of Performance Paradox and Clinical Integration Challenges.Journal of Korean Neurosurgical Society · 2026Article
- The Reliability of Human Evaluation of Large Language Models in Health Care Settings: Scoping Review.Journal of medical Internet research · 2026Article
- Are large language models such as ChatGPT, capable of supporting patients and general practitioners after spine surgery?European spine journal : official publication of the European Spine Society, the European Spinal Deformity Society, and the European Section of the Cervical Spine Research Society · 2026Article
- Large language models in spine care and research.European spine journal : official publication of the European Spine Society, the European Spinal Deformity Society, and the European Section of the Cervical Spine Research Society · 2026Review
- Publicly Accessible Large Language Model Responses to Frequently Asked Questions About Spondylodiscitis: Preliminary Expert Evaluation.Journal of medical Internet research · 2026Article
- Pain-responsive to digital self-management in lumbar disc herniation: the mediating role of patient activation and the moderating effect of pain intensity.BMC health services research · 2026Article
- Comparative Quality Assessment of Artificial Intelligence in Patient Education on Platelet-Rich Plasma (PRP) Therapy.Journal of personalized medicine · 2026Article
- Application and efficacy of artificial intelligence in patient education on spinal cord injuries.European spine journal : official publication of the European Spine Society, the European Spinal Deformity Society, and the European Section of the Cervical Spine Research Society · 2026Article
- "Can We Trust Them?" An Expert Evaluation of Large Language Models to Provide Sleep and Jet Lag Recommendations for Athletes.Sports medicine (Auckland, N.Z.) · 2026Article
- Evaluating the role of large language models in supporting patient education during the informed consent process for routine radiology procedures.The British journal of radiology · 2025Article
- Human evaluation of large language models in healthcare: gaps, challenges, and the need for standardization.npj health systems · 2025Article
- Current trends and future prospects of language models and processing systems in spine surgery - a scoping review.Neurosurgical review · 2025Article
- Assessing the efficacy of large language models in spinal health information dissemination.Brain & spine · 2024Article
- Large language models in patient education: a scoping review of applications in medicine.Frontiers in medicine · 2024Article
- Evaluating Accuracy and Readability of Responses to Midlife Health Questions: A Comparative Analysis of Six Large Language Model Chatbots.Journal of mid-life healthArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
9 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Introduction: Generative AI is revolutionizing patient education in healthcare, particularly through chatbots that offer personalized, clear medical information. Reliability and accuracy are vital in AI-driven patient education. Research question: How effective are Large Language Models (LLM), such as ChatGPT and Google Bard, in delivering accurate and understandable patient education on lumbar disc herniation? Material and methods: Ten Frequently Asked Questions about lumbar disc herniation were selected from 133 questions and were submitted to three LLMs. Six experienced spine surgeons rated the responses on a scale from "excellent" to "unsatisfactory," and evaluated the answers for exhaustiveness, clarity, empathy, and length. Statistical analysis involved Fleiss Kappa, Chi-square, and Friedman tests. Results: Out of the responses, 27.2% were excellent, 43.9% satisfactory with minimal clarification, 18.3% satisfactory with moderate clarification, and 10.6% unsatisfactory. There were no significant differences in overall ratings among the LLMs (p = 0.90); however, inter-rater reliability was not achieved, and large differences among raters were detected in the distribution of answer frequencies. Overall, ratings varied among the 10 answers (p = 0.043). The average ratings for exhaustiveness, clarity, empathy, and length were above 3.5/5. Discussion and conclusion: LLMs show potential in patient education for lumbar spine surgery, with generally positive feedback from evaluators. The new EU AI Act, enforcing strict regulation on AI systems, highlights the need for rigorous oversight in medical contexts. In the current study, the variability in evaluations and occasional inaccuracies underline the need for continuous improvement. Future research should involve more advanced models to enhance patient-physician communication.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.