ArticleJournal of the American Medical Informatics Association : JAMIA2020
The 2019 National Natural language processing (NLP) Clinical Challenges (n2c2)/Open Health NLP (OHNLP) shared task on clinical concept normalization for clinical records.
Article in Journal of the American Medical Informatics Association : JAMIA, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 23 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
23 citing papers in PubMed.
- RxMap: an LLM-assisted tool for medication normalization.JAMIA open · 2026Article
- A neural-symbolic AI agent system for biomedical concept mapping.NPJ digital medicine · 2026Article
- Development and content validation of a health assessment item bank for regional health big data: a sequential mixed-methods approach.Health and quality of life outcomes · 2026Article
- Article
- Processing of Short-Form Content in Clinical Narratives: Systematic Scoping Review.Journal of medical Internet research · 2024Article
- Multi-head CRF classifier for biomedical multi-class named entity recognition on Spanish clinical notes.Database : the journal of biological databases and curation · 2024Article
- Towards discovery: an end-to-end system for uncovering novel biomedical relations.Database : the journal of biological databases and curation · 2024Article
- End-to-end pseudonymization of fine-tuned clinical BERT models : Privacy preservation with maintained data utility.BMC medical informatics and decision making · 2024Article
- Impact of Translation on Biomedical Information Extraction: Experiment on Real-Life Clinical Notes.JMIR medical informatics · 2024Article
- BELB: a biomedical entity linking benchmark.Bioinformatics (Oxford, England) · 2023Review
- Chemical identification and indexing in full-text articles: an overview of the NLM-Chem track at BioCreative VII.Database : the journal of biological databases and curation · 2023Article
- Clinical Artificial Intelligence: Design Principles and Fallacies.Clinics in laboratory medicine · 2023Review
- An artificial intelligence-based approach for identifying rare disease patients using retrospective electronic health records applied for Pompe disease.Frontiers in neurology · 2023Article
- Standardizing Multi-site Clinical Note Titles to LOINC Document Ontology: A Transformer-based Approach.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2023Article
- Linguistic and ontological challenges of multiple domains contributing to transformed health ecosystems.Frontiers in medicine · 2023Article
- A scoping review of publicly available language tasks in clinical natural language processing.Journal of the American Medical Informatics Association : JAMIA · 2022Article
- Year 2021: COVID-19, Information Extraction and BERTization among the Hottest Topics in Medical Natural Language Processing.Yearbook of medical informatics · 2022Article
- Chemical identification and indexing in PubMed full-text articles using deep learning and heuristics.Database : the journal of biological databases and curation · 2022Article
- A simple neural vector space model for medical concept normalization using concept embeddings.Journal of biomedical informatics · 2022Article
- Improving broad-coverage medical entity linking with semantic type prediction and large-scale datasets.Journal of biomedical informatics · 2021Article
Corrections and comments
- Erratum issued
Authors and funding
4 authors.
Funding
Abstract
objectiveThe 2019 National Natural language processing (NLP) Clinical Challenges (n2c2)/Open Health NLP (OHNLP) shared task track 3, focused on medical concept normalization (MCN) in clinical records. This track aimed to assess the state of the art in identifying and matching salient medical concepts to a controlled vocabulary. In this paper, we describe the task, describe the data set used, compare the participating systems, present results, identify the strengths and limitations of the current state of the art, and identify directions for future research. MATERIALS AND
methodsParticipating teams were provided with narrative discharge summaries in which text spans corresponding to medical concepts were identified. This paper refers to these text spans as mentions. Teams were tasked with normalizing these mentions to concepts, represented by concept unique identifiers, within the Unified Medical Language System. Submitted systems represented 4 broad categories of approaches: cascading dictionary matching, cosine distance, deep learning, and retrieve-and-rank systems. Disambiguation modules were common across all approaches.
resultsA total of 33 teams participated in the MCN task. The best-performing team achieved an accuracy of 0.8526. The median and mean performances among all teams were 0.7733 and 0.7426, respectively.
conclusionsOverall performance among the top 10 teams was high. However, several mention types were challenging for all teams. These included mentions requiring disambiguation of misspelled words, acronyms, abbreviations, and mentions with more than 1 possible semantic type. Also challenging were complex mentions of long, multi-word terms that may require new ways of extracting and representing mention meaning, the use of domain knowledge, parse trees, or hand-crafted rules.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.