ArticleJournal of neurointerventional surgery2025
Evaluating local open-source large language models for data extraction from unstructured reports on mechanical thrombectomy in patients with ischemic stroke.
Article in Journal of neurointerventional surgery, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 14 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
14 citing papers in PubMed.
- Prognostic data extraction harnessing a privacy-preserving large language model: a clinician-AI collaborative retrospective evaluation in head and neck oncology.NPJ precision oncology · 2026Article
- Comparing performance of seven fine-tuned open-source large language models in summarizing and predicting outcome-relevant information from mechanical thrombectomy reports in patients with acute ischemic stroke.European radiology · 2026Article
- AI-driven Medical Care: Evaluation of Large Language Models in Generating Personalized Stroke Education Materials.Healthcare informatics research · 2026Article
- Machine learning models powered by emergency medical services data enhance stroke triage in prehospital settings.Scientific reports · 2026Article
- COPE: Chain-of-Thought Prediction Engine for open-source large language model-based stroke outcome prediction from clinical notes.BMJ digital health & AI · 2026Article
- Predicting treatment retention in medication for opioid use disorder: a machine learning approach using NLP and LLM-derived clinical features.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- Evaluating the diagnostic accuracy of vision language models for neuroradiological image interpretation.NPJ digital medicine · 2025Article
- Predicting Acute Cerebrovascular Events in Stroke Alerts Using Large-Language Models and Structured Data.medRxiv : the preprint server for health sciences · 2025Article
- Article
- Llama 3.1 405B Is Comparable to GPT-4 for Extraction of Data from Thrombectomy Reports-A Step Towards Secure Data Extraction.Clinical neuroradiology · 2025Article
- Automated MRI protocoling in neuroradiology in the era of large language models.La Radiologia medica · 2025Article
- Current Landscape and Future Directions Regarding Generative Large Language Models in Stroke Care: Scoping Review.JMIR medical informatics · 2025Article
- Data extraction from free-text stroke CT reports using GPT-4o and Llama-3.3-70B: the impact of annotation guidelines.European radiology experimental · 2025Article
- Large Language Model Applications for Health Information Extraction in Oncology: Scoping Review.JMIR cancer · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
13 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundA study was undertaken to assess the effectiveness of open-source large language models (LLMs) in extracting clinical data from unstructured mechanical thrombectomy reports in patients with ischemic stroke caused by a vessel occlusion.
methodsWe deployed local open-source LLMs to extract data points from free-text procedural reports in patients who underwent mechanical thrombectomy between September 2020 and June 2023 in our institution. The external dataset was obtained from a second university hospital and comprised consecutive cases treated between September 2023 and March 2024. Ground truth labeling was facilitated by a human-in-the-loop (HITL) approach, with time metrics recorded for both automated and manual data extractions. We tested three models-Mixtral, Qwen, and BioMistral-assessing their performance on precision, recall, and F1 score across 15 clinical categories such as National Institute of Health Stroke Scale (NIHSS) scores, occluded vessels, and medication details.
resultsThe study included 1000 consecutive reports from our primary institution and 50 reports from a secondary institution. Mixtral showed the highest precision, achieving 0.99 for first series time extraction and 0.69 for occluded vessel identification within the internal dataset. In the external dataset, precision ranged from 1.00 for NIHSS scores to 0.70 for occluded vessels. Qwen showed moderate precision with a high of 0.85 for NIHSS scores and a low of 0.28 for occluded vessels. BioMistral had the broadest range of precision, from 0.81 for first series times to 0.14 for medication details. The HITL approach yielded an average time savings of 65.6% per case, with variations from 45.95% to 79.56%.
conclusionThis study highlights the potential of using LLMs for automated clinical data extraction from medical reports. Incorporating HITL annotations enhances precision and also ensures the reliability of the extracted data. This methodology presents a scalable privacy-preserving option that can significantly support clinical documentation and research endeavors.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.