ArticleJAMA network open2025
Accuracy of Artificial Intelligence vs Professionally Translated Discharge Instructions.
Article in JAMA network open, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 10 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
10 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Language-related eligibility criteria in UK randomised trials: a systematic review of extended research reports.Trials · 2026Pooled it
- Translation approaches to support systemic anti-cancer therapy consent for individuals with limited English proficiency.Supportive care in cancer : official journal of the Multinational Association of Supportive Care in Cancer · 2026Trial
- Randomized Controlled Trial Comparing AI-Generated, Clinician-Edited to Human-Generated After-Visit Summaries.Mayo Clinic proceedings. Digital health · 2026Article
- Integrating Nonpharmacologic Strategies for Pain with Inclusion, Respect, and Equity (INSPIRE): Cultural and linguistic tailoring for a digital health intervention.Internet interventions · 2026Article
- Prioritizing Equity in Design and Implementation of Consumer-Facing Digital Resources to Support Engagement and Shared Decision-Making.Journal of medical Internet research · 2026Article
- Wearable Real-Time Translation in Rhinology: A Comparative Study of Accuracy and Efficiency.International forum of allergy & rhinology · 2026Article
- Disparate language and model effects on AI-based translation and recognition of genetic conditions.Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Article
- Beyond translation: A review into preserving human connection in AI-mediated health language interpretation.PLOS digital health · 2026Article
- Implementing artificial intelligence (AI)-supported communication tools in healthcare: System-level perspectives.Digital healthArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors.
Funding
Abstract
Importance: Patients using languages other than English are a group at risk of poor health outcomes and encounter barriers to access of translated written materials. Although artificial intelligence (AI) may offer an opportunity to improve access, few studies have evaluated the accuracy and safety of AI translation for clinical care under routine practice conditions. Objective: To investigate the accuracy of AI translation compared with professional human translation of patient-specific issued pediatric inpatient discharge instructions. Design, Setting, and Participants: This comparative effectiveness analysis compared translations by a neural machine translation model vs professional translators using patient-specific pediatric inpatient discharge instructions received by families between May 18, 2023, and May 18, 2024, at a single center academic pediatric hospital. Instructions were translated to Simplified Chinese, Somali, Spanish, and Vietnamese by professional translators and the Azure AI system and then broken into scoring sections. Two professional translators per language evaluated translations (blinded to source) on an established 5-point scale for fluency, adequacy, meaning, and error severity, with 1 indicating worst performance and 5 indicating best performance. Exposure: AI vs professional translation. Main Outcome and Measure: Quality of discharge instruction translation, including fluency, adequacy, meaning, and severity of errors. Results: A total of 148 sections from 34 discharge instructions were analyzed. When considering all 4 languages together, average fluency, adequacy, and meaning were lower among AI compared with professional human translations. Among all tested languages, mean (SD) fluency for AI translations was 2.98 (1.12) compared with 3.90 (0.96) for professional translations (difference, 0.92; 95% CI, 0.83-1.01; P < .001), adequacy was 3.81 (1.14) compared with 4.56 (0.70) (difference, 0.74; 95% CI, 0.65-0.83; P < .001), meaning was 3.38 (1.15) compared with 4.28 (0.84) (difference, 0.90; 95% CI, 0.80-0.99; P < .001), and error severity was 3.53 (1.28) compared with 4.48 (0.88) (difference, 0.95; 95% CI, 0.85-1.06; P < .001). Compared with professional translations, the Spanish AI translations were noninferior in adequacy (difference, 0.08; 95% CI, -0.02 to 0.19) and error severity (difference, 0.03; 95% CI, -0.09 to 0.14) but inferior in fluency (difference, 0.38; 95% CI, 0.23-0.53) and just crossed the inferiority threshold in meaning (difference, 0.08; 95% CI, -0.04 to 0.20). The Chinese, Vietnamese, and Somali AI translations were inferior to the professional translations across all metrics, with the greatest differences for Somali. Conclusions and Relevance: In this comparative effectiveness analysis of AI- vs professionally translated issued discharge instructions, AI-translated instructions performed similarly for Spanish but worse for other languages tested. Validation and clinical implementation of AI-based translation will require special attention to languages of lesser diffusion to prevent creating new inequities.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.