Evidence map›Paper›PMID 41772676›Full record

ArticleJournal of neuroengineering and rehabilitation2026

Benchmarking large language models against human experts in rehabilitation medicine: a multidimensional evaluation.

Wenhui Cao, Mengjian Qu, Tao Zhu, Jing Liu, Ying Shen, Jihua Zou, Yi Li, Haiming Wang, Lisha Zhang, Huifang Liu and 7 more

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Journal of neuroengineering and rehabilitation, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1 paper.

0numbers the graph read from it
0cells of the map it votes in
1citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

1 citing paper in PubMed.

  1. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

17 authors.

Wenhui Cao *Department of Rehabilitation, The First Affiliated Hospital, Hengyang Medical School, University of South China, Hengyang, 421001, Hunan, China.
Mengjian Qu *Department of Rehabilitation, The First Affiliated Hospital, Hengyang Medical School, University of South China, Hengyang, 421001, Hunan, China.
Tao ZhuSchool of Computer Science, University of South China, Hengyang, 421001, Hunan, China.
Jing LiuDepartment of Rehabilitation, The First Affiliated Hospital, Hengyang Medical School, University of South China, Hengyang, 421001, Hunan, China.
Ying ShenDepartment of Rehabilitation Medicine, The First Affiliated Hospital with Nanjing Medical University, Nanjing, 210029, Jiangsu, China.
Jihua ZouDepartment of Rehabilitation Medicine, Zhujiang Hospital, Southern Medical University, Guangzhou, 510280, Guangdong, China.
Yi LiDepartment of Rehabilitation Medicine, Rehabilitation Medicine Center, Rehabilitation Key Laboratory of Sichuan Province, West China Hospital, Sichuan University, Chengdu, 610041, Sichuan, China.
Haiming WangRehabilitation Medicine Department, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, 450052, Henan, China.
Lisha ZhangFaculty of Health and Social Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China.
Huifang LiuDepartment of Rehabilitation Medicine, Sichuan Provincial People's Hospital, School of Medicine, , University of Electronic Science and Technology of China, Chengdu, 610072, Sichuan, China.
Qi WuDepartment of Rehabilitation, The First Affiliated Hospital, Hengyang Medical School, University of South China, Hengyang, 421001, Hunan, China.
Guijuan ZhouDepartment of Rehabilitation, The First Affiliated Hospital, Hengyang Medical School, University of South China, Hengyang, 421001, Hunan, China.
Guanghua SunDepartment of Rehabilitation, The First Affiliated Hospital, Hengyang Medical School, University of South China, Hengyang, 421001, Hunan, China.
Helin GongSJTU Paris Elite Institute of Technology, Shanghai Jiao Tong University, Shanghai, 200240, China.
Yaping WanSchool of Computer Science, University of South China, Hengyang, 421001, Hunan, China. ypwan@aliyun.com.
Xiaofeng HeSchool of Computer Science, University of South China, Hengyang, 421001, Hunan, China. hxf@usc.edu.cn.
Jun ZhouDepartment of Rehabilitation, The First Affiliated Hospital, Hengyang Medical School, University of South China, Hengyang, 421001, Hunan, China. zhoujun8005@163.com.

Funding

The First Affiliated Hospital of University of South China IRP-M&E-2025-09
6 · The paper itself

Abstract

backgroundRehabilitation medicine faces a significant challenge due to the rising demand for services coupled with a shortage of specialized professionals. Large Language Models (LLMs) show promise for enhancing clinical efficiency, but their evaluation has been largely limited to simulated scenarios, lacking direct performance comparisons with human experts in complex, real-world clinical tasks.

objectiveTo systematically benchmark five state-of-the-art LLMs against senior physiatrists in formulating comprehensive rehabilitation plans for authentic clinical cases, evaluating their utility as clinical decision support tools.

methodsWe conducted a rigorous, blinded evaluation using 48 authentic cases across six subspecialties. Plans generated by five LLMs (Grok-4, Gemini−2.5-pro, ChatGPT-5-2025-08-07, Deepseek-r1-0528, and Claude-opus-4-20250514) were compared with expert-authored plans. A panel of 6 senior physiatrists evaluated the plans using a multi-dimensional framework covering four key domains: Clinical Applicability and Safety (primary safety endpoint), Scientific Rigor, Individualization, and Clarity. To address the data’s hierarchical structure, we employed Linear Mixed-Effects Models (LMM) with random intercepts for cases and raters, and fixed effects for models and language. Pairwise comparisons were adjusted using the Holm-Bonferroni correction.

resultsQuantitative analysis revealed that Grok-4 (mean 4.31) and Gemini−2.5-pro (mean 4.14) significantly outperformed the human benchmark (derived from standardized expert solutions) (mean 3.56; [Formula: see text]). Notably, the open-source Deepseek-r1 (mean 3.69) also achieved a statistically significant advantage over experts ([Formula: see text]). Conversely, human experts scored numerically higher than Claude-opus-4 (mean 3.50), though this difference was not statistically significant ([Formula: see text]). Qualitative analysis further highlighted human experts’ distinct strengths in strategic pathway design and humanistic care.

conclusionsTop-tier LLMs demonstrate capability in generating high-quality, evidence-based plans, positioning them as effective “executors” for drafting preliminary regimens. We propose a human-AI collaboration paradigm where experts function as “strategists,” focusing on optimization and humanistic care to elevate rehabilitation service quality.

Indexed as

BenchmarkingDecision Support Systems, ClinicalLarge Language ModelsRehabilitationHumansBenchmarkingClinical decision supportHuman-AI collaborationLarge language modelsMultidimensional evaluation frameworkPerformance evaluationRehabilitation medicineRehabilitation plan

Identifiers

PMID41772676
PMCPMC12951902

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.