Evidence map›Paper›PMID 41274285›Full record

ArticleCell reports. Medicine2025

Multi-center benchmarking of large language models for clinical decision support in lung cancer screening.

Zhiyun Duan, Xu Huang, Rong Lu, Wenyi Xu, Huaning Liu, Yucai Geng, Natsuki Takahashi, Ying Wu, Qingle Wang, Yang Song and 5 more

Abstract readMulticenter Study
PubMed Full text at PMC Publisher
In one paragraph

Article in Cell reports. Medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 2 papers.

0numbers the graph read from it
0cells of the map it votes in
2citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

2 citing papers in PubMed.

  1. Article
  2. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

15 authors.

Zhiyun DuanDepartment of Thoracic Surgery, Zhongshan Hospital, Fudan University, Shanghai, China; Intelligent Medicine Institute, Fudan University, Shanghai, China.
Xu HuangDepartment of Thoracic Surgery, Zhongshan Hospital, Fudan University, Shanghai, China.
Rong LuDepartment of Radiology, Huashan Hospital, Fudan University, Shanghai, China.
Wenyi XuDepartment of Thoracic Surgery, Zhongshan Hospital, Fudan University, Shanghai, China.
Huaning LiuDepartment of Statistics, University of Illinois Urbana-Champaign, Champaign, IL, USA.
Yucai GengDepartment of Immunology, School of Basic Medical Sciences, Shanghai Medical College, Fudan University, Shanghai, China.
Natsuki TakahashiSchool of Clinical Medicine, Shanghai Medical College, Fudan University, Shanghai, China.
Ying WuSchool of Clinical Medicine, Shanghai Medical College, Fudan University, Shanghai, China.
Qingle WangDepartment of Radiology, Zhongshan Hospital, Fudan University, Shanghai, China.
Yang SongMR Research Collaboration Team, Siemens Healthineers Ltd, Shanghai, China.
Hongbo XuDepartment of Cardiothoracic Surgery, Luan Affiliated Hospital of Anhui Medical University, Luan, China.
Han TangDepartment of Thoracic Surgery, Zhongshan Hospital, Fudan University, Shanghai, China.
Fei LanShanghai Key Laboratory of Medical Epigenetics, State International Co-laboratory of Medical Epigenetics and Metabolism, Institutes of Biomedical Sciences, Fudan University, Shanghai, China; Key Laboratory of Carcinogenesis and Cancer Invasion, Ministry of Education, Liver Cancer Institute, Zhongshan Hospital, Fudan University, Shanghai, China.
Roland EilsIntelligent Medicine Institute, Fudan University, Shanghai, China; Zhongshan Hospital, Fudan University, Shanghai, China; Center for Digital Health, Berlin Institute of Health (BIH), Charite University Medicine Berlin, Berlin, Germany. Electronic address: roland_eils@fudan.edu.cn.
Lijie TanDepartment of Thoracic Surgery, Zhongshan Hospital, Fudan University, Shanghai, China; Department of Thoracic Surgery, Zhongshan Hospital Xiamen Branch, Fudan University, Xiamen, China. Electronic address: tan.lijie@zs-hospital.sh.cn.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Large language models (LLMs) are increasingly explored for clinical applications, but their ability to generate management recommendations for lung cancer screening remains uncertain. In this cross-sectional, multi-center study, 148 anonymized low-dose computed tomography (CT) reports from three healthcare institutions are used to assess the readability, accuracy, and consistency of four widely adopted models (GPT-3.5, GPT-4, Claude 3 Sonnet, and Claude 3 Opus). Among them, Claude 3 Opus produces the most readable recommendations, while GPT-4 achieves the highest clinical accuracy. Importantly, performance dose not differ significantly across institutions, underscoring the robustness of these models to variations in reporting templates and their utility in diverse healthcare settings. In an exploratory analysis, two state-of-the-art models, proprietary GPT-4o and its open-source counterpart DeepSeek-R1, show comparable performance to GPT-4, outperforming GPT-3.5. These findings highlight the potential role of LLMs to enhance clinical decision support in lung cancer screening across diverse healthcare settings.

Indexed as

BenchmarkingDecision Support Systems, ClinicalEarly Detection of CancerLanguageLung NeoplasmsCross-Sectional StudiesFemaleHumansLarge Language ModelsMaleMiddle AgedTomography, X-Ray ComputedAI for healthclinical decision supportlarge language modelsLLMslung cancer screening

Identifiers

PMID41274285
PMCPMC12765833

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.