ArticleJournal of translational medicine2025
Temporal evolution of large language models (LLMs) in oncology.
Article in Journal of translational medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 5 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
5 citing papers in PubMed.
- AI-Powered Clinical Decision Support in Dentistry: Comparative Evaluation of Large Language Models for Oral Medicine and Periodontal Diagnosis.Medical science monitor : international medical journal of experimental and clinical research · 2026Article
- Medicine digital transformation: evidence from Chinese physicians on generative artificial intelligence implementation and challenges.Journal of translational medicine · 2026Article
- Comparative assessment of quality, consistency, and reference accuracy of MIH-related clinical information generated by ChatGPT-4o and DeepSeek R1.BMC oral health · 2026Article
- Patient education for neuromyelitis optica spectrum disorder using large language models: combining expert assessment and real-world patient interaction.Frontiers in physiology · 2026Article
- Accuracy of large language models in head and neck cancers: a comparative analysis of ChatGPT and Gemini in TNM staging and clinical decision support.Frontiers in oncology · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
20 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundLarge language models (LLMs) are increasingly being applied in healthcare; however, their performance in specialized fields, such as oncology, is subject to temporal factors, including knowledge decay and concept drift. The impact of these temporal dynamics on LLM question-answering accuracy in oncology remains inadequately evaluated. This study aims to systematically assess the temporal evolution of LLM accuracy in responding to oncology-related questions using real-world data.
methodWe systematically collected relevant literature through 2025 by searching LLM-related keywords in PubMed, Google Scholar, and Web of Science databases. The inclusion criteria were as follows: (1) cancer-related research; (2) clear and complete question descriptions; and (3) complete answers. The final sample (n = 23) contained 614 research questions, comprising subjective questions (n = 223) and multiple-choice questions (n = 391). Following randomization of responses generated by three LLMs (ChatGPT-3.5, ChatGPT-4, and Gemini), we evaluated their accuracy across different cancer categories using both original scoring criteria and Likert scale scoring methods. Data analysis was performed using R statistical software, employing random or fixed effects models to calculate pooled mean differences (MD) and relative risks (RR) with their 95% confidence intervals (CI).
resultsThe findings demonstrated that in both subjective and objective oncology assessments, ChatGPT-3.5 (subjective questions MD = -3.30; objective questions RR = 0.92) and ChatGPT-4 (subjective questions MD = -7.17; objective questions RR = 0.93) showed declining performance trends over time, while Gemini exhibited significant improvements over time (subjective questions MD = 11.48; objective questions RR = 1.15). Notably, ChatGPT-3.5's performance on subjective questions revealed a significant turning point between March 14, 2023, and April 26, 2023, shifting from initially superior performance on newer questions to inferior performance compared with original questions, with the performance gap progressively widening.
conclusionsOur meta-analysis reveals temporal performance degradation in ChatGPT-3.5 and ChatGPT-4, which contrasts with the consistent improvement observed in Gemini. These findings provide essential guidance for the evidence-based deployment of LLMs in oncology.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.