ArticleJournal of the American Medical Informatics Association : JAMIA2019
Enhancing clinical concept extraction with contextual embeddings.
Article in Journal of the American Medical Informatics Association : JAMIA, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 95 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
95 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Application of Artificial Intelligence Methods to Pharmacy Data for Cancer Surveillance and Epidemiology Research: A Systematic Review.JCO clinical cancer informatics · 2020Pooled it
- Automated Extraction of Postoperative Cancer Recurrence and Metastasis From Computed Tomography (CT) Reports: Semisupervised Deep Learning Study.JMIR medical informatics · 2026Article
- A Secure, Scalable Large Language Model-Based System (CIDER) for High-Throughput Clinical Data Extraction From Medical Reports: Retrospective Validation Study.Journal of medical Internet research · 2026Article
- A study of large language models for patient information extraction: Model architecture, fine-tuning strategy, and multi-task instruction tuning.Journal of biomedical informatics · 2026Article
- Life events extraction from healthcare notes for veteran acute suicide risk prediction.Journal of the American Medical Informatics Association : JAMIA · 2026Article
- A Sentence Classification-Based Medical Status Extraction Pipeline for Electronic Health Records: Institutional Case Study.JMIR medical informatics · 2026Article
- Improving Retrieval Augmented Generation for Health Care by Fine-Tuning Clinical Embedding Models: Development and Evaluation Study.Journal of medical Internet research · 2026Article
- Advocating the potential of artificial intelligence for syndrome discovery in syndromic surveillance systems: A scoping review.iScience · 2026Review
- Information extraction from clinical notes: are we ready to switch to large language models?Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Leveraging natural language processing and machine learning to identify chronic conditions from primary care electronic medical records.Scientific reports · 2026Article
- Structured-to-text ClinicalBERT embeddings with random Forest for heart disease prediction: a proof-of-concept study on the UCI Statlog dataset.Frontiers in artificial intelligence · 2026Article
- Article
- Extracting TNFi switching reasons and trajectories from real-world data using large language models.JAMIA open · 2025Article
- Large Language Models in Critical Care Medicine: Scoping Review.JMIR medical informatics · 2025Article
- Evaluation of SURUS: a named entity recognition NLP system to extract knowledge from interventional study records.BMC medical research methodology · 2025Article
- A patient-centered approach to developing and validating a natural language processing model for extracting patient-reported symptoms.Scientific reports · 2025Article
- Predicting Drug-Side Effect Relationships From Parametric Knowledge Embedded in Biomedical BERT Models: Methodological Study With a Natural Language Processing Approach.JMIR medical informatics · 2025Article
- Utility of word embeddings from large language models in medical diagnosis.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- Approaches for extracting daily dosage from free-text prescription signatures in heart failure with reduced ejection fraction: a comparative study.JAMIA open · 2025Article
- Enhancing Cross-Domain Generalizability in Social Determinants of Health Extraction with Prompt-Tuning Large Language Models.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2025Article
35 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
Abstract
objectiveNeural network-based representations ("embeddings") have dramatically advanced natural language processing (NLP) tasks, including clinical NLP tasks such as concept extraction. Recently, however, more advanced embedding methods and representations (eg, ELMo, BERT) have further pushed the state of the art in NLP, yet there are no common best practices for how to integrate these representations into clinical tasks. The purpose of this study, then, is to explore the space of possible options in utilizing these new models for clinical concept extraction, including comparing these to traditional word embedding methods (word2vec, GloVe, fastText). MATERIALS AND
methodsBoth off-the-shelf, open-domain embeddings and pretrained clinical embeddings from MIMIC-III (Medical Information Mart for Intensive Care III) are evaluated. We explore a battery of embedding methods consisting of traditional word embeddings and contextual embeddings and compare these on 4 concept extraction corpora: i2b2 2010, i2b2 2012, SemEval 2014, and SemEval 2015. We also analyze the impact of the pretraining time of a large language model like ELMo or BERT on the extraction performance. Last, we present an intuitive way to understand the semantic information encoded by contextual embeddings.
resultsContextual embeddings pretrained on a large clinical corpus achieves new state-of-the-art performances across all concept extraction tasks. The best-performing model outperforms all state-of-the-art methods with respective F1-measures of 90.25, 93.18 (partial), 80.74, and 81.65.
conclusionsWe demonstrate the potential of contextual embeddings through the state-of-the-art performance these methods achieve on clinical concept extraction. Additionally, we demonstrate that contextual embeddings encode valuable semantic information not accounted for in traditional word representations.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.