ArticleJournal of biomedical semantics2022
We are not ready yet: limitations of state-of-the-art disease named entity recognizers.
Article in Journal of biomedical semantics, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 7 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
7 citing papers in PubMed.
- BioTriplex: a full-text annotated corpus for fine-tuning language models in gene-disease relation extraction tasks.Bioinformatics (Oxford, England) · 2026Article
- Robust privacy amidst innovation with large language models through a critical assessment of the risks.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- Enhancing medical coding efficiency through domain-specific fine-tuned large language models.npj health systems · 2025Article
- Integrating deep learning architectures for enhanced biomedical relation extraction: a pipeline approach.Database : the journal of biological databases and curation · 2024Article
- Dataset of miRNA-disease relations extracted from textual data using transformer-based neural networks.Database : the journal of biological databases and curation · 2024Article
- Review
- AIONER: all-in-one scheme-based biomedical named entity recognition using deep learning.Bioinformatics (Oxford, England) · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundIntense research has been done in the area of biomedical natural language processing. Since the breakthrough of transfer learning-based methods, BERT models are used in a variety of biomedical and clinical applications. For the available data sets, these models show excellent results - partly exceeding the inter-annotator agreements. However, biomedical named entity recognition applied on COVID-19 preprints shows a performance drop compared to the results on test data. The question arises how well trained models are able to predict on completely new data, i.e. to generalize.
resultsBased on the example of disease named entity recognition, we investigate the robustness of different machine learning-based methods - thereof transfer learning - and show that current state-of-the-art methods work well for a given training and the corresponding test set but experience a significant lack of generalization when applying to new data.
conclusionsWe argue that there is a need for larger annotated data sets for training and testing. Therefore, we foresee the curation of further data sets and, moreover, the investigation of continual learning processes for machine learning-based models.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.