Evidence map›Paper›PMID 42151588›Full record

ArticleJournal of cancer research and clinical oncology2026

Evaluating reasoning models for therapy recommendations in gastrointestinal stromal tumors: expert and LLM-based evaluations of OpenAI o1 and DeepSeek-R1.

Melissa Harbrücker, Franka Menge, Arshia Taebi, Dominik Nörenberg, Tobias Speer, Christoph Reißfelder, Peter Hohenberger, Jens Jakob, Chengpeng Li, Cui Yang

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Journal of cancer research and clinical oncology, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1 paper.

0numbers the graph read from it
0cells of the map it votes in
1citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

1 citing paper in PubMed.

  1. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Melissa HarbrückerDepartment of Surgery, Medical Faculty Mannheim, University Medical Center Mannheim, University of Heidelberg, Mannheim, Germany.
Franka MengeDepartment of Surgery, Medical Faculty Mannheim, University Medical Center Mannheim, University of Heidelberg, Mannheim, Germany.
Arshia TaebiDepartment of Surgery, Medical Faculty Mannheim, University Medical Center Mannheim, University of Heidelberg, Mannheim, Germany.
Dominik NörenbergDepartment of Radiology and Nuclear Medicine, Medical Faculty Mannheim, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany.
Tobias SpeerIndependent Researcher, Ludwigshafen Am Rhein, Germany.
Christoph ReißfelderDepartment of Surgery, Medical Faculty Mannheim, University Medical Center Mannheim, University of Heidelberg, Mannheim, Germany.
Peter HohenbergerDivision of Surgical Oncology & Thoracic Surgery, Department of Surgery, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany.
Jens JakobDepartment of Surgery, Medical Faculty Mannheim, University Medical Center Mannheim, University of Heidelberg, Mannheim, Germany.
Chengpeng LiKey Laboratory of Carcinogenesis and Translational Research (Ministry of Education/Beijing), Sarcoma Center, Peking University Cancer Hospital & Institute, Beijing, China. chengpengli@bjmu.edu.cn.ORCID http://orcid.org/0000-0001-6751-2770
Cui YangDepartment of Surgery, Medical Faculty Mannheim, University Medical Center Mannheim, University of Heidelberg, Mannheim, Germany. Cui.Yang@medma.uni-heidelberg.de.ORCID http://orcid.org/0000-0003-3779-0424

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

purposeThis study aims to evaluate two advanced reasoning LLMs in generating treatment recommendations for real-world gastrointestinal stromal tumor (GIST) cases and assess their concordance with multidisciplinary team (MDT) decisions at a certified tertiary sarcoma center.

methodsSixty-five real-world GIST cases from a tertiary sarcoma center were used to compare two advanced reasoning models-OpenAI o1 and DeepSeek-R1. Recommendations were generated using a multi-expert prompting strategy with current clinical guidelines as context. Five sarcoma specialists and an independent LLM (Mistral AI) evaluated alignment with MDT decisions and guideline concordance.

resultsOpenAI o1 achieved higher concordance with MDT decisions than DeepSeek-R1 (76.9% vs. 53.8%, p < 0.001) and more recommendations aligned with either MDT or guidelines (80.0% vs. 63.1%, p = 0.005). Inter-rater reliability among human evaluators was excellent (ICC = 0.929). The LLM judge's evaluations showed moderate agreement with human assessments (κ = 0.647). OpenAI o1 responses were significantly longer than those of DeepSeek-R1 and MDT records.

conclusionsOpenAI o1 outperformed DeepSeek-R1 in generating clinically relevant GIST therapy recommendations. The study highlights the feasibility of using LLMs both as decision support tools and as evaluators ("LLM-as-a-judge") in oncology, while emphasizing the need for expert oversight in clinical deployment.

Indexed as

Clinical Decision-MakingGastrointestinal NeoplasmsGastrointestinal Stromal TumorsFemaleHumansLarge Language ModelsMaleArtificial intelligenceDecision-makingGastrointestinal stromal tumorHuman-in-the-loopLarge language modelMultidisciplinary

Identifiers

PMID42151588
PMCPMC13357472

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.