ArticleCommunications medicine2025
Comprehensive testing of large language models for extraction of structured data in pathology.
Article in Communications medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 22 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
22 citing papers in PubMed.
- The consensus-based CINEX guideline for reporting clinical information extraction studies.Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Automated Identification of Complex Percutaneous Coronary Intervention from Cardiac Catheterization Reports using Large Language Models.medRxiv : the preprint server for health sciences · 2026Article
- A Secure, Scalable Large Language Model-Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study.Journal of medical Internet research · 2026Article
- Article
- Adapting transformer-based encoder models for automated billing code assignment of current procedural terminology from pathology reports in multi-institutional settings.Journal of pathology informatics · 2026Article
- Large language models enable prognostic stratification of cancer patients using real-world clinical notes.PLOS digital health · 2026Article
- Benchmarking large language models for cell-free RNA diagnostic biomarker discovery.Nature communications · 2026Article
- An agentic framework for autonomous scientific discovery in cancer pathology.Nature medicine · 2026Article
- Open-source large language model-based on-premises pipeline for automated data extraction from unstructured electronic health records: a pilot study.BMJ health & care informatics · 2026Article
- Development and External Validation of a Large Language Model-Based Clinical Decision-Support System for Colonoscopy Surveillance Intervals.Clinical gastroenterology and hepatology : the official clinical practice journal of the American Gastroenterological Association · 2026Article
- Digital Registrar: A Schema-First Framework for Multi-Cancer Privacy-Preserving Pathology Abstraction via Local LLMs.Diagnostics (Basel, Switzerland) · 2026Article
- Multimodal Prediction of Renal Tumor Malignancy From Radiology Reports and Structured Electronic Health Records: Retrospective Cohort Study.JMIR medical informatics · 2026Article
- BI-RADS-compliant structured mammography reporting using locally deployed large language models under privacy constraints.European radiology · 2026Article
- Assessing the performance of multimodal large language models in experimental information extraction from liquid-liquid phase separation literature.Scientific reports · 2026Article
- Leveraging Large Language Models to Extract Prognostic Pathology Features in Ewing Sarcoma.bioRxiv : the preprint server for biology · 2026Article
- Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics.Briefings in bioinformatics · 2026Article
- Toward Artificial Intelligence in Oncology and Cardiology: A Narrative Review of Systems, Challenges, and Opportunities.Journal of clinical medicine · 2025Article
- Combining Clinician Expertise with Prompt Engineering enhances Small Language Models Reliability for Cancer Entity Recognition in Electronic Health Records.medRxiv : the preprint server for health sciences · 2025Article
- Large Language Model and Knowledge Graph-Driven AJCC Staging of Prostate Cancer Using Pathology Reports.Diagnostics (Basel, Switzerland) · 2025Article
- Computational pathology in bladder cancer: A scoping review.Bladder cancer (Amsterdam, Netherlands)Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
Abstract
backgroundPathology departments generate large volumes of unstructured data as free-text diagnostic reports. Converting these reports into structured formats for analytics or artificial intelligence projects requires substantial manual effort by specialized personnel. While recent studies show promise in using advanced language models for structuring pathology data, they primarily rely on proprietary models, raising cost and privacy concerns. Additionally, important aspects such as prompt engineering and model quantization for deployment on consumer-grade hardware remain unaddressed.
methodsWe created a dataset of 579 annotated pathology reports in German and English versions. Six language models (proprietary: GPT-4; open-source: Llama2 13B, Llama2 70B, Llama3 8B, Llama3 70B, and Qwen2.5 7B) were evaluated for their ability to extract eleven key parameters from these reports. Additionally, we investigated model performance across different prompt engineering strategies and model quantization techniques to assess practical deployment scenarios.
resultsHere we show that open-source language models extract structured data from pathology reports with high precision, matching the accuracy of proprietary GPT-4 model. The precision varies significantly across different models and configurations. These variations depend on specific prompt engineering strategies and quantization methods used during model deployment.
conclusionsOpen-source language models demonstrate comparable performance to proprietary solutions in structuring pathology report data. This finding has significant implications for healthcare institutions seeking cost-effective, privacy-preserving data structuring solutions. The variations in model performance across different configurations provide valuable insights for practical deployment in pathology departments. Our publicly available bilingual dataset serves as both a benchmark and a resource for future research.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.