ArticleJournal of the American Medical Informatics Association : JAMIA2023
Clinical concept and relation extraction using prompt-based machine reading comprehension.
Article in Journal of the American Medical Informatics Association : JAMIA, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 18 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
18 citing papers in PubMed, 1 synthesis or guideline pooled it, 28 citations in OpenAlex.
- Applications of Natural Language Processing and Large Language Models for Social Determinants of Health: Systematic Review.Journal of medical Internet research · 2026Pooled it
- A study of large language models for patient information extraction: Model architecture, fine-tuning strategy, and multi-task instruction tuning.Journal of biomedical informatics · 2026Article
- Leveraging Large Language Models in Extracting Drug Safety Information from Prescription Drug Labels.Drug safety · 2026Article
- Clinical temporal relation extraction with long-context transformers: a robustness study on MIMIC-III and MIMIC-IV.Frontiers in digital health · 2026Article
- Domain and Language adaptive pre-training of BERT models for Korean-English bilingual clinical text analysis.BMC medical informatics and decision making · 2025Article
- Development of a natural language processing algorithm to extract social determinants of health from clinician notes.American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons · 2025Article
- Decoding substance use disorder severity from clinical notes using a large language model.Npj mental health research · 2025Article
- Twenty-Five Years of Evolution and Hurdles in Electronic Health Records and Interoperability in Medical Research: Comprehensive Review.Journal of medical Internet research · 2025Review
- Enhancing Cross-Domain Generalizability in Social Determinants of Health Extraction with Prompt-Tuning Large Language Models.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2025Article
- Automatic Summarization of Doctor-Patient Encounter Dialogues Using Large Language Model through Prompt Tuning.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2025Article
- Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need.Journal of the American Medical Informatics Association : JAMIA · 2024Article
- Extracting Pulmonary Nodules and Nodule Characteristics from Radiology Reports of Lung Cancer Screening Patients Using Transformer Models.Journal of healthcare informatics research · 2024Article
- Year 2023 in Biomedical Natural Language Processing: a Tribute to Large Language Models and Generative AI.Yearbook of medical informatics · 2024Article
- Model tuning or prompt Tuning? a study of large language models for clinical concept and relation extraction.Journal of biomedical informatics · 2024Article
- Question-answering system extracts information on injection drug use from clinical notes.Communications medicine · 2024Article
- Opportunities and Risks of Large Language Models in Psychiatry.NPP - digital psychiatry and neuroscience · 2024Article
- Narrative Feature or Structured Feature? A Study of Large Language Models to Identify Cancer Patients at Risk of Heart Failure.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2024Article
- A study of generative large language model for medical research and healthcare.NPJ digital medicine · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors at 2 institutions in 1 country.
Funding
Abstract
objectiveTo develop a natural language processing system that solves both clinical concept extraction and relation extraction in a unified prompt-based machine reading comprehension (MRC) architecture with good generalizability for cross-institution applications.
methodsWe formulate both clinical concept extraction and relation extraction using a unified prompt-based MRC architecture and explore state-of-the-art transformer models. We compare our MRC models with existing deep learning models for concept extraction and end-to-end relation extraction using 2 benchmark datasets developed by the 2018 National NLP Clinical Challenges (n2c2) challenge (medications and adverse drug events) and the 2022 n2c2 challenge (relations of social determinants of health [SDoH]). We also evaluate the transfer learning ability of the proposed MRC models in a cross-institution setting. We perform error analyses and examine how different prompting strategies affect the performance of MRC models. RESULTS AND
conclusionThe proposed MRC models achieve state-of-the-art performance for clinical concept and relation extraction on the 2 benchmark datasets, outperforming previous non-MRC transformer models. GatorTron-MRC achieves the best strict and lenient F1-scores for concept extraction, outperforming previous deep learning models on the 2 datasets by 1%-3% and 0.7%-1.3%, respectively. For end-to-end relation extraction, GatorTron-MRC and BERT-MIMIC-MRC achieve the best F1-scores, outperforming previous deep learning models by 0.9%-2.4% and 10%-11%, respectively. For cross-institution evaluation, GatorTron-MRC outperforms traditional GatorTron by 6.4% and 16% for the 2 datasets, respectively. The proposed method is better at handling nested/overlapped concepts, extracting relations, and has good portability for cross-institute applications. Our clinical MRC package is publicly available at https://github.com/uf-hobi-informatics-lab/ClinicalTransformerMRC.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.