ArticleJournal of the American Medical Informatics Association : JAMIA2020
2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records.
Article in Journal of the American Medical Informatics Association : JAMIA, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 122 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
122 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Transforming epilepsy research: A systematic review on natural language processing applications.Epilepsia · 2023Pooled it
- Hierarchical prompting with reasoning large language models for immune checkpoint inhibitor-associated myocarditis evidence extraction pilot study.European heart journal. Digital health · 2026Article
- A Large Language Model for Extracting Post-marketing Adverse Drug Events from Clinical Notes in the Electronic Health Record.Drug safety · 2026Article
- Multilingual Evidence-Based Question-Answering for Stroke Discharge Summaries: Study of Cross-Lingual Heterogeneity in Clinical Reports.Journal of medical Internet research · 2026Article
- Using Natural Language Processing to Identify Adverse Drug Events Characterized by Medication Replacement in Primary Care Electronic Medical Records: Algorithm and Validation Study.Journal of medical Internet research · 2026Article
- The word and the way: strategies for domain-specific BERT pre-training in German medical NLP.BMC medical informatics and decision making · 2026Article
- Relation extraction for diet, non-communicable disease and biomarker associations (RECoDe): a CoDiet study.Bioinformatics (Oxford, England) · 2026Article
- The Evaluation of Transformer Models for the Detection of Adverse Drug Events: A Benchmark Study Using Dutch Free-Text Documents of Hospitalized Patients.Drug safety · 2026Article
- A study of large language models for patient information extraction: Model architecture, fine-tuning strategy, and multi-task instruction tuning.Journal of biomedical informatics · 2026Article
- GLiNER-BioMed: a suite of efficient models for open biomedical named entity recognition.Bioinformatics (Oxford, England) · 2026Article
- AI-Based Automation for Medication Reconciliation: Scoping Review.Journal of medical Internet research · 2026Article
- Enhanced Adverse-Event Detection and Drug-Event Relation Extraction from Clinical Notes.medRxiv : the preprint server for health sciences · 2026Article
- Ontology-Based Medication Named Entity Recognition Using Pretrained Transformer Models From a Thai Hospital: Model Fine-Tuning and Validation Study.JMIR formative research · 2026Article
- Information extraction from clinical notes: are we ready to switch to large language models?Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Holistic evaluation of large language models for medical tasks with MedHELM.Nature medicine · 2026Article
- Evaluating large language models for clinical note processing: local fine-tuning and internal-external validation using electronic health records from South Asia.BMC medical informatics and decision making · 2026Article
- Leveraging Large Language Models in Extracting Drug Safety Information from Prescription Drug Labels.Drug safety · 2026Article
- Transformer-based relation extraction and concept normalization using an annotated clinical trials corpus.Scientific data · 2026Article
- Article
- Curation and Extraction of Drug-Related Entities from Reddit Platform.Proceedings. IEEE International Conference on Healthcare Informatics · 2026Article
62 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
Abstract
objectiveThis article summarizes the preparation, organization, evaluation, and results of Track 2 of the 2018 National NLP Clinical Challenges shared task. Track 2 focused on extraction of adverse drug events (ADEs) from clinical records and evaluated 3 tasks: concept extraction, relation classification, and end-to-end systems. We perform an analysis of the results to identify the state of the art in these tasks, learn from it, and build on it. MATERIALS AND
methodsFor all tasks, teams were given raw text of narrative discharge summaries, and in all the tasks, participants proposed deep learning-based methods with hand-designed features. In the concept extraction task, participants used sequence labelling models (bidirectional long short-term memory being the most popular), whereas in the relation classification task, they also experimented with instance-based classifiers (namely support vector machines and rules). Ensemble methods were also popular.
resultsA total of 28 teams participated in task 1, with 21 teams in tasks 2 and 3. The best performing systems set a high performance bar with F1 scores of 0.9418 for concept extraction, 0.9630 for relation classification, and 0.8905 for end-to-end. However, the results were much lower for concepts and relations of Reasons and ADEs. These were often missed because local context is insufficient to identify them.
conclusionsThis challenge shows that clinical concept extraction and relation classification systems have a high performance for many concept types, but significant improvement is still required for ADEs and Reasons. Incorporating the larger context or outside knowledge will likely improve the performance of future systems.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.