Implementation of an AI-Assisted Workflow for Rehabilitation Discharge Summary Creation: Evaluation of Documentation Efficiency, Usability, and Hallucinations in a 290-Bed General Hospital in Japan.

Background The widespread adoption of electronic health records (EHRs) has imposed substantial documentation burdens on healthcare workers and is internationally recognized as a major contributor to clinician burnout. Rehabilitation therapists also experience documentation burdens comparable to those of physicians and nurses. However, integrated evaluations of the efficiency, usability, and safety of generative artificial intelligence (AI)-assisted creation of rehabilitation discharge summaries remain limited. Objective This study aimed to evaluate the impact of an AI-assisted rehabilitation discharge summary creation workflow - combining a commercial generative AI service with an in-house Excel/Visual Basic for Applications (VBA) macro-based transcription system (Microsoft Corp., Redmond, Washington) - on documentation efficiency, usability, and safety in the rehabilitation department of a general hospital in Japan. Methods This single-center, before-after implementation study was conducted in the rehabilitation department of a 290-bed general hospital in Kyoto, Japan. Documentation time for rehabilitation discharge summary creation (pre-implementation, n=16; post-implementation, n=22; therapists, n=21) was analyzed using a linear mixed-effects model with random intercepts for therapist ID, supplemented by the Wilcoxon signed-rank test. Usability was assessed using the System Usability Scale (SUS) administered to 18 cooperating therapists (physical, occupational, and speech-language therapists) following implementation, with results interpreted according to the established criteria. Hallucination and omission evaluation was performed on 11 randomly sampled AI-generated rehabilitation discharge summaries by two independent raters using a published clinical safety framework; disagreements were resolved through consensus discussion. Results The median documentation time decreased from 23.0 minutes pre-implementation to 10.0 minutes post-implementation. The adjusted linear mixed-effects model revealed a significant reduction of 12.26 minutes (95% CI: -15.71 to -8.81; p<0.001). The Wilcoxon signed-rank test in the nine therapists who contributed to both periods also confirmed a large effect size (r=0.87, p=0.014). No difference in documentation time was observed based on previous lecture attendance (β=0.05, p=0.98). The median SUS score was 83.75 (IQR: 72.5-90.0), corresponding to Letter Grade B (80-89), with 15 of 18 respondents (83.3%) classified as "Acceptable" (≥70). In the hallucination and omission evaluation, 9 of 11 rehabilitation discharge summaries (81.8%) contained at least one error, yielding a total of 19 hallucinations and 1 omission. Major errors accounted for 52.6% (10/19), and the major rate of fabrication-type hallucinations was 83.3% (5/6). Conclusions This exploratory, single-center implementation study found that an AI-assisted rehabilitation discharge summary workflow was associated with reduced documentation time and favorable usability ratings. However, hallucinations occurred at a level requiring therapist verification and correction, with clinically relevant errors in high-risk content segments. These findings provide preliminary evidence that a low-cost, low-technical-barrier AI-assisted documentation workflow may improve documentation efficiency and usability within the studied institutional context. Safe implementation requires final human verification and risk-stratified review of high-risk content segments.