ArticleBMC bioinformatics2012
A corpus of full-text journal articles is a robust evaluation tool for revealing differences in performance of biomedical natural language processing tools.
Article in BMC bioinformatics, 2012. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 41 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
41 citing papers in PubMed.
- CellExLink: End-to-end cell-type recognition and normalization in biomedical text.PLoS computational biology · 2026Article
- Cell phenotypes in the biomedical literature: a systematic analysis and text mining corpus.bioRxiv : the preprint server for biology · 2026Article
- BioTriplex: a full-text annotated corpus for fine-tuning language models in gene-disease relation extraction tasks.Bioinformatics (Oxford, England) · 2026Article
- GeneCup: mining PubMed and GWAS catalog for gene-keyword relationships.G3 (Bethesda, Md.) · 2022Article
- Parallel sequence tagging for concept recognition.BMC bioinformatics · 2022Article
- Examining linguistic shifts between preprints and publications.PLoS biology · 2022Article
- ECO-CollecTF: A Corpus of Annotated Evidence-Based Assertions in Biomedical Manuscripts.Frontiers in research metrics and analytics · 2021Article
- Do medicine and cell biology talk to each other? A study of vocabulary similarities between fields.Brazilian journal of medical and biological research = Revista brasileira de pesquisas medicas e biologica · 2021Article
- Dependency parsing of biomedical text with BERT.BMC bioinformatics · 2020Article
- ProtFus: A Comprehensive Method Characterizing Protein-Protein Interactions of Fusion Proteins.PLoS computational biology · 2019Article
- Doc2Hpo: a web application for efficient and accurate HPO concept curation.Nucleic acids research · 2019Article
- From POS tagging to dependency parsing for biomedical event extraction.BMC bioinformatics · 2019Article
- Article
- Configurable web-services for biomedical document annotation.Journal of cheminformatics · 2018Article
- Textpresso Central: a customizable platform for searching, text mining, viewing, and curating biomedical literature.BMC bioinformatics · 2018Article
- Semantic annotation of consumer health questions.BMC bioinformatics · 2018Article
- Gene ontology concept recognition using named concept: understanding the various presentations of the gene functions in biomedical literature.Database : the journal of biological databases and curation · 2018Article
- Entity recognition in the biomedical domain using a hybrid approach.Journal of biomedical semantics · 2017Article
- Coreference annotation and resolution in the Colorado Richly Annotated Full Text (CRAFT) corpus of biomedical journal articles.BMC bioinformatics · 2017Article
- Gold-standard ontology-based anatomical annotation in the CRAFT Corpus.Database : the journal of biological databases and curation · 2017Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
15 authors.
Funding
Abstract
backgroundWe introduce the linguistic annotation of a corpus of 97 full-text biomedical publications, known as the Colorado Richly Annotated Full Text (CRAFT) corpus. We further assess the performance of existing tools for performing sentence splitting, tokenization, syntactic parsing, and named entity recognition on this corpus.
resultsMany biomedical natural language processing systems demonstrated large differences between their previously published results and their performance on the CRAFT corpus when tested with the publicly available models or rule sets. Trainable systems differed widely with respect to their ability to build high-performing models based on this data.
conclusionsThe finding that some systems were able to train high-performing models based on this corpus is additional evidence, beyond high inter-annotator agreement, that the quality of the CRAFT corpus is high. The overall poor performance of various systems indicates that considerable work needs to be done to enable natural language processing systems to work well when the input is full-text journal articles. The CRAFT corpus provides a valuable resource to the biomedical natural language processing community for evaluation and training of new models for biomedical full text publications.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.