ArticleJournal of the American Medical Informatics Association : JAMIA2015
A multilingual gold-standard corpus for biomedical concept recognition: the Mantra GSC.
Article in Journal of the American Medical Informatics Association : JAMIA, 2015. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 18 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
18 citing papers in PubMed.
- Transformer-based relation extraction and concept normalization using an annotated clinical trials corpus.Scientific data · 2026Article
- A roadmap for medical large language models: a review of foundations, applications, and challenges.Military Medical Research · 2026Review
- Review
- Article
- Annotation-preserving machine translation of English corpora to validate Dutch clinical concept extraction tools.Journal of the American Medical Informatics Association : JAMIA · 2024Article
- Improving biomedical entity linking for complex entity mentions with LLM-based text simplification.Database : the journal of biological databases and curation · 2024Article
- Impact of Translation on Biomedical Information Extraction: Experiment on Real-Life Clinical Notes.JMIR medical informatics · 2024Article
- Cancer-Alterome: a literature-mined resource for regulatory events caused by genetic alterations in cancer.Scientific data · 2024Article
- An overview of biomedical entity linking throughout the years.Journal of biomedical informatics · 2023Review
- DrNote: An open medical annotation service.PLOS digital health · 2022Article
- MedTAG: a portable and customizable annotation tool for biomedical documents.BMC medical informatics and decision making · 2021Article
- A clinical trials corpus annotated with UMLS entities to enhance the access to evidence-based medicine.BMC medical informatics and decision making · 2021Article
- ECCParaCorp: a cross-lingual parallel corpus towards cancer education, dissemination and application.BMC medical informatics and decision making · 2020Article
- Automated Risk Assessment for School Violence: a Pilot Study.The Psychiatric quarterly · 2018Article
- Clinical Natural Language Processing in languages other than English: opportunities and challenges.Journal of biomedical semantics · 2018Review
- Annotation of phenotypes using ontologies: a gold standard for the training and evaluation of natural language processing systems.Database : the journal of biological databases and curation · 2018Article
- Semantic annotation in biomedicine: the current landscape.Journal of biomedical semantics · 2017Review
- Collaborative relation annotation and quality analysis in Markyt environment.Database : the journal of biological databases and curation · 2017Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveTo create a multilingual gold-standard corpus for biomedical concept recognition. MATERIALS AND
methodsWe selected text units from different parallel corpora (Medline abstract titles, drug labels, biomedical patent claims) in English, French, German, Spanish, and Dutch. Three annotators per language independently annotated the biomedical concepts, based on a subset of the Unified Medical Language System and covering a wide range of semantic groups. To reduce the annotation workload, automatically generated preannotations were provided. Individual annotations were automatically harmonized and then adjudicated, and cross-language consistency checks were carried out to arrive at the final annotations.
resultsThe number of final annotations was 5530. Inter-annotator agreement scores indicate good agreement (median F-score 0.79), and are similar to those between individual annotators and the gold standard. The automatically generated harmonized annotation set for each language performed equally well as the best annotator for that language. DISCUSSION: The use of automatic preannotations, harmonized annotations, and parallel corpora helped to keep the manual annotation efforts manageable. The inter-annotator agreement scores provide a reference standard for gauging the performance of automatic annotation techniques.
conclusionTo our knowledge, this is the first gold-standard corpus for biomedical concept recognition in languages other than English. Other distinguishing features are the wide variety of semantic groups that are being covered, and the diversity of text genres that were annotated.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.