Evidence map›Paper›PMID 40633961›Full record

ArticleBMJ quality & safety2026

Evaluation of the accuracy and safety of machine translation of patient-specific discharge instructions: a comparative analysis.

Marianna Kong, Alicia Fernandez, Jaskaran Bains, Ana Milisavljevic, Katherine C Brooks, Akash Shanmugam, Leslie Avilez, Junhong Li, Vladyslav Honcharov, Andersen Yang and 1 more

Abstract readComparative Study
PubMed Full text at PMC Publisher
In one paragraph

Article in BMJ quality & safety, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 18 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
18citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

18 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Trial
  4. Article
  5. Article
  6. Article
  7. Article
  8. Observational
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Review
  16. Review
  17. Review
  18. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

11 authors.

Marianna KongDepartment of Family and Community Medicine, University of California San Francisco, San Francisco, California, USA.ORCID http://orcid.org/0000-0002-2276-4250
Alicia FernandezDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA.ORCID http://orcid.org/0000-0002-7975-6614
Jaskaran BainsDepartment of Emergency Medicine, University of California San Francisco, San Francisco, California, USA.
Ana MilisavljevicDivision of Hospital Medicine, University of California San Francisco, San Francisco, California, USA.
Katherine C BrooksDivision of Hospital Medicine, Zuckerberg San Francisco General, University of California San Francisco, San Francisco, California, USA.ORCID http://orcid.org/0000-0003-1905-6963
Akash ShanmugamUniversity of California San Francisco School of Medicine, San Francisco, California, USA.
Leslie AvilezDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA.
Junhong LiDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA.ORCID http://orcid.org/0009-0003-6477-8989
Vladyslav HoncharovDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA.ORCID http://orcid.org/0000-0002-6045-6515
Andersen YangDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA.ORCID http://orcid.org/0009-0008-4446-0849
Elaine C KhoongDivision of General Internal Medicine at Zuckerberg San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA Elaine.Khoong@ucsf.edu.ORCID http://orcid.org/0000-0002-2514-3572

Funding

Designing a Remote Monitoring Intervention for Value & Equity in Hypertension (DRIVE-HTN)K23HL157750 · NHLBI · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI KHOONG, ELAINE · 2021 to 2025
$921k
NHLBI NIH HHS K23 HL157750
6 · The paper itself

Abstract

introductionMachine translation of patient-specific information could mitigate language barriers if sufficiently accurate and non-harmful and may be particularly useful in healthcare encounters when professional translators are not readily available. We evaluated the translation accuracy and potential for harm of ChatGPT-4 and Google Translate in translating from English to Spanish, Chinese and Russian.

methodsWe used ChatGPT-4 and Google Translate to translate 50 sets (316 sentences) of deidentified, patient-specific, clinician free-text emergency department instructions into Spanish, Chinese and Russian. These were then back-translated into English by professional translators and double-coded by physicians for accuracy and potential for clinical harm.

resultsAt the sentence level, we found that both tools were ≥90% accurate in translating English to Spanish (accuracy: GPT 97%, Google Translate 96%) and English to Chinese (accuracy: GPT 95%; Google Translate 90%); neither tool performed as well in translating English to Russian (accuracy: GPT 89%; Google Translate 80%). At the instruction set level, 16%, 24% and 56% of Spanish, Chinese and Russian GPT-translated instruction sets contained at least one inaccuracy. For Google Translate, 24%, 56% and 66% of Spanish, Chinese and Russian translations contained at least one inaccuracy. The potential for harm due to inaccurate translations was ≤1% for both tools in all languages at the sentence level and ≤6% at the instruction set level. GPT was significantly more accurate than Google Translate in Chinese and Russian at the sentence level; the potential for harm was similar.

conclusionThese results support the potential of machine translation tools to mitigate gaps in translation services for low-stakes written communication from English to Spanish, while also strengthening the case for caution and for professional oversight in non-low-risk communication. Further research is needed to evaluate machine translation for other languages and more technical content.

Indexed as

Patient DischargeTranslatingCommunication BarriersHumansLanguagePatient SafetyCommunicationEmergency departmentInformation technologyPatient-centred carePatient education

Identifiers

PMID40633961
PMCPMC12252260

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.