ArticleCommunications medicine2025
Large language model comparisons between English and Chinese query performance for cardiovascular prevention.
Article in Communications medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 10 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
10 citing papers in PubMed.
- Multilingual Disparities in Large Language Model-Based Symptom Detection for Global Disease Surveillance: Evaluation Study.JMIR AI · 2026Article
- Quality, completeness, readability, and response time of AI chatbots in the management of deep caries and pulp exposure: a bilingual comparative study.BMC oral health · 2026Article
- ChatGPT-4's Consistency, Specificity, and Inclusion of Behavior Change Techniques in Delivering Smoking Cessation Advice in Traditional Chinese: A Content Analysis.Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco · 2026Article
- Evaluation of ChatGPT-4o in oral and maxillofacial surgery examinations: a comparative study of performance on U.S. dental decks and chinese dental licensing examination practice questions.BMC oral health · 2026Article
- Deciphering AI-chatbots utilization in clinical practice, research and education among Chinese medical students in generative AI era: a cross-sectional study.BMC medical education · 2026Article
- Evolving Consultation: Enhancing Ophthalmic Diagnostic Performance Using Large Language Model.Ophthalmology science · 2026Article
- Large Language Models in Patient Health Communication for Atherosclerotic Cardiovascular Disease: Pilot Cross-Sectional Comparative Analysis.JMIR medical informatics · 2026Article
- From algorithms to operating room: can large language models master China's attending anesthesiology exam? A cross-sectional evaluation.International journal of surgery (London, England) · 2026Article
- Construction and evaluation of the knowledge graph and large model question-answering system for Jin San Zhen therapy: a tool study for primary care and general practice.Frontiers in medicine · 2026Article
- Towards A Fair Duel: Reflections on the Evaluation of DeepSeek-R1 and ChatGPT-4o in Chinese Medical Education.Journal of medical systems · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
17 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundLarge language model (LLM) offer promise in addressing layperson queries related to cardiovascular disease (CVD) prevention. However, the accuracy and consistency of information provided by current general LLMs remain unclear.
methodsWe evaluated capabilities of BARD (Google's bidirectional language model for semantic understanding), ChatGPT-3.5, ChatGPT-4.0 (OpenAI's conversational models for generating human-like text) and ERNIE (Baidu's knowledge-enhanced language model for context understanding) in addressing CVD prevention queries in English and Chinese. 75 CVD prevention questions were posed to each LLM. The primary outcome was the accuracy of responses (rated as appropriate, borderline, inappropriate).
resultsFor English prompts, the chatbots' appropriate ratings are as follows: BARD at 88.0%, ChatGPT-3.5 at 92.0%, and ChatGPT-4.0 at 97.3%. All models demonstrate temporal improvement in initially suboptimal responses, with BARD and ChatGPT-3.5 each improving by 67% (6/9 and 4/6), and ChatGPT-4.0 achieving a 100% (2/2) improvement rate. Both BARD and ChatGPT-4.0 outperform ChatGPT-3.5 in recognizing the correctness of their responses. For Chinese prompts, the "appropriate" ratings are: ERNIE at 84.0%, ChatGPT-3.5 at 88.0%, and ChatGPT-4.0 at 85.3%. However, ERNIE outperform ChatGPT-3.5 and ChatGPT-4.0 in temporal improvement and self-awareness of correctness.
conclusionsFor CVD prevention queries in English, ChatGPT-4.0 outperforms other LLMs in generating appropriate responses, temporal improvement, and self-awareness. The LLMs' performance drops slightly for Chinese queries, reflecting potential language bias in these LLMs. Given growing availability and accessibility of LLM chatbots, regular and rigorous evaluations are essential to thoroughly assess the quality and limitations of the medical information they provide across widely spoken languages.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.