ArticleAMIA ... Annual Symposium proceedings. AMIA Symposium2012
Building gold standard corpora for medical natural language processing tasks.
Article in AMIA ... Annual Symposium proceedings. AMIA Symposium, 2012. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 43 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
43 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Using Natural Language Processing to Measure and Improve Quality of Diabetes Care: A Systematic Review.Journal of diabetes science and technology · 2021Pooled it
- Hierarchical prompting with reasoning large language models for immune checkpoint inhibitor-associated myocarditis evidence extraction pilot study.European heart journal. Digital health · 2026Article
- From narrative evidence to computable knowledge: a decision-relevant corpus for medicinal herb-disease relationships.Journal of biomedical semantics · 2026Article
- RoBuster-Corpus Annotated With Risk of Bias Text Spans in Randomized Controlled Trials in Physiotherapy and Rehabilitation: Corpus Development and Annotation Study.JMIR formative research · 2026Article
- Quantifying Global Foreign Affairs with a Multimodal Dataset of Diplomatic Websites.Scientific data · 2025Article
- Clinical Information Extraction From Notes of Veterans With Lymphoid Malignancies: Natural Language Processing Study.JMIR medical informatics · 2025Article
- Dementia-related volumetric assessments in neuroradiology reports: a natural language processing-based study.BMJ open · 2025Article
- Evaluation of SURUS: a named entity recognition NLP system to extract knowledge from interventional study records.BMC medical research methodology · 2025Article
- Evaluating generative AI for qualitative data extraction in community-based fisheries management literature.Environmental evidence · 2025Article
- Sentences, entities, and keyphrases extraction from consumer health forums using multi-task learning.Journal of biomedical semantics · 2025Article
- RiCoRecA:Frontiers in artificial intelligence · 2025Article
- Contextualized race and ethnicity annotations for clinical text from MIMIC-III.Scientific data · 2024Article
- Annotation of epilepsy clinic letters for natural language processing.Journal of biomedical semantics · 2024Article
- A pseudonymized corpus of occupational health narratives for clinical entity recognition in Spanish.BMC medical informatics and decision making · 2024Article
- Europe's Largest Research Infrastructure for Curated Medical Data Models with Semantic Annotations.Methods of information in medicine · 2024Article
- Toward Clinical-Grade Evaluation of Large Language Models.International journal of radiation oncology, biology, physics · 2024Article
- Exploring the Latest Highlights in Medical Natural Language Processing across Multiple Languages: A Survey.Yearbook of medical informatics · 2023Article
- Representation of Pain Concepts and Terms in Existing Ontologies and Taxonomies.Pain medicine (Malden, Mass.) · 2023Article
- Trustworthy assertion classification through prompting.Journal of biomedical informatics · 2022Article
- SemClinBr - a multi-institutional and multi-specialty semantically annotated corpus for Portuguese clinical NLP tasks.Journal of biomedical semantics · 2022Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
9 authors.
Funding
Abstract
We present the construction of three annotated corpora to serve as gold standards for medical natural language processing (NLP) tasks. Clinical notes from the medical record, clinical trial announcements, and FDA drug labels are annotated. We report high inter-annotator agreements (overall F-measures between 0.8467 and 0.9176) for the annotation of Personal Health Information (PHI) elements for a de-identification task and of medications, diseases/disorders, and signs/symptoms for information extraction (IE) task. The annotated corpora of clinical trials and FDA labels will be publicly released and to facilitate translational NLP tasks that require cross-corpora interoperability (e.g. clinical trial eligibility screening) their annotation schemas are aligned with a large scale, NIH-funded clinical text annotation project.
Indexed as
Identifiers
23304283PMC3540456What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.