ArticleNPJ digital medicine2025
Large language model agents can use tools to perform clinical calculations.
Article in NPJ digital medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 27 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
27 citing papers in PubMed.
- Integrating Large Language Models Into Trauma Education for Medical Students: Randomized Controlled Pilot Trial.JMIR medical education · 2026Trial
- AI in Proton Therapy Treatment Planning: A Narrative Review.International journal of particle therapy · 2026Article
- Skill-guided agentic reasoning enables longitudinal synthesis of myeloma records at expert-level concordance.NPJ precision oncology · 2026Article
- Patient-governed agentic health data exchange.npj health systems · 2026Review
- Artificial intelligence in hepatocellular carcinoma diagnosis and AI-assisted treatment support: Current landscape and future directions.Liver research (Beijing, China) · 2026Review
- Utilization of a HIPAA-compliant large language model chatbot in an academic pediatric medical center.PLOS digital health · 2026Article
- Development of an LLM pipeline exceeding physician-documented cardiovascular risk scores under routine clinical conditions.European heart journal. Digital health · 2026Article
- Behavior Change Content and Implementation of Large Language Model-Driven Conversational Agents in Cardiometabolic Care: Scoping Review.Journal of medical Internet research · 2026Article
- Integrating Generative AI into Nomograms for Breast Cancer Nodal Risk Predictions.Annals of surgical oncology · 2026Article
- Artificial intelligence reshaping the paradigm of hematologic malignancy diagnosis and treatment: From static assessment to dynamic precision management.Annals of hematology · 2026Review
- Artificial intelligence agents in cancer research and oncology.Nature reviews. Cancer · 2026Review
- SPARROW: subtyping Parkinson's disease with agentic reasoning and robust omics workflow.NPJ artificial intelligence · 2026Article
- Performance of large language models in neonatal resuscitation assessments versus healthcare providers: an exploratory study.Frontiers in artificial intelligence · 2026Article
- Use of AI agents to assess preoperative frailty in cancer patients.npj digital surgery · 2026Article
- Generative Artificial Intelligence in Healthcare: Automation Bias, Deskilling, and Cognitive Implications - A Systematic Review.Journal of healthcare leadership · 2026Review
- Preliminary evaluation of DeepSeek-R1 and GPT-5.3 in selected PET/CT clinical scenarios: patient preparation, report interpretation, and diagnostic reasoning.Frontiers in medicine · 2026Article
- Are artificial intelligence (AI) agents ready for medicine and biomedical research? A narrative review.Journal of Yeungnam medical science · 2026Article
- Asking the Right Questions: Benchmarking Large Language Models in the Development of Clinical Consultation Templates.Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing · 2026Article
- Promises and challenges of applying large language models in the healthcare domain.Frontiers in digital health · 2026Review
- A multi-agent approach to neurological clinical reasoning.PLOS digital health · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
Abstract
Large language models (LLMs) can answer expert-level questions in medicine but are prone to hallucinations and arithmetic errors. Early evidence suggests LLMs cannot reliably perform clinical calculations, limiting their potential integration into clinical workflows. We evaluated ChatGPT's performance across 48 medical calculation tasks, finding incorrect responses in one-third of trials (n = 212). We then assessed three forms of agentic augmentation: retrieval-augmented generation, a code interpreter tool, and a set of task-specific calculation tools (OpenMedCalc) across 10,000 trials. Models with access to task-specific tools showed the greatest improvement, with LLaMa and GPT-based models demonstrating a 5.5-fold (88% vs 16%) and 13-fold (64% vs 4.8%) reduction in incorrect responses, respectively, compared to the unimproved models. Our findings suggest that integration of machine-readable, task-specific tools may help overcome LLMs' limitations in medical calculations.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.