ArticleBioinformatics (Oxford, England)2020
BioBERT: a pre-trained biomedical language representation model for biomedical text mining.
Article in Bioinformatics (Oxford, England), 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1,778 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
1,778 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- Machine learning-assisted mRNA vaccine pharmacovigilance: a systematic review of multi-source real-world data.BMC medical informatics and decision making · 2026Pooled it
- Recent Advances in AI for Automated ICD Coding: A Systematic Literature Review.Journal of medical systems · 2026Pooled it
- Microbial named entity recognition and normalisation for AI-assisted literature review and meta-analysis.Bioinformatics (Oxford, England) · 2026Pooled it
- Article
- Benchmarking commercial large language models for gene-disease-phenotype extraction from full-text human genetics literature.Quantitative biology (Beijing, China) · 2026Article
- Fine-grained evaluation of a domain-specific Q&A dataset to support trustworthy medical language models.Health information science and systems · 2026Article
- The Rise of Small Language Models in Healthcare: A Comprehensive Survey.Computer science review · 2026Article
- AI-based multimodal integration of genomics and electronic health records.Nature reviews. Genetics · 2026Review
- SPELL: A scalable NLP method using regular expressions and large language models for clinical information extraction.Computer methods and programs in biomedicine · 2026Article
- Multimodal learning enables chat-based exploration of single-cell data.Nature biotechnology · 2026Article
- Optimizing generative artificial intelligence for clinical summarization: a blinded comparison study of automated versus human care planning synopses.JAMIA open · 2026Article
- Large language model-based triage to identify antiretroviral therapy adherence barriers and risk levels in patient messages.JAMIA open · 2026Article
- Protocol for tri-modal pretraining and downstream fine-tuning of protein representations using ProteinAligner.STAR protocols · 2026Article
- Machine Learning to Prioritize High-Severity Patient Safety Events for Institutional Investigation: Algorithm Development and Validation Study.JMIR medical informatics · 2026Article
- A Generalizable Multimodal Model for Treatment-Stratified Risk and Survival Assessment under Real-World Constraints: A Multi-Center Study of Colorectal Cancer.Advanced science (Weinheim, Baden-Wurttemberg, Germany) · 2026Article
- Antibody-drug conjugate engineering: from design to efficacy and safety.Signal transduction and targeted therapy · 2026Review
- Large Language Models for Clinical Data Extraction in Workplace Injury Rehabilitation: Protocol for a Retrospective Pilot Study of Accuracy, Fairness, and Methodological Considerations in Workers' Compensation Medical Chart Review.JMIR research protocols · 2026Article
- CDELDA: A Content-Based Dual-Encoder for Cold-Start lncRNA-Disease Association Prediction.Biomedicines · 2026Article
- High-Recall Biomedical Language Models for Radiation Oncology Evidence Synthesis: An Integrated Systematic Review of Prediction Models for Radiotherapy-Induced Complications in Nasopharyngeal Carcinoma.Life (Basel, Switzerland) · 2026Review
- A text mining and ontology-based approach using phenotypes to obtain relevant literature for rare diseases.iScience · 2026Article
1,718 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
motivationBiomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical literature has gained popularity among researchers, and deep learning has boosted the development of effective biomedical text mining models. However, directly applying the advancements in NLP to biomedical text mining often yields unsatisfactory results due to a word distribution shift from general domain corpora to biomedical corpora. In this article, we investigate how the recently introduced pre-trained language model BERT can be adapted for biomedical corpora.
resultsWe introduce BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora. With almost the same architecture across tasks, BioBERT largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre-trained on biomedical corpora. While BERT obtains performance comparable to that of previous state-of-the-art models, BioBERT significantly outperforms them on the following three representative biomedical text mining tasks: biomedical named entity recognition (0.62% F1 score improvement), biomedical relation extraction (2.80% F1 score improvement) and biomedical question answering (12.24% MRR improvement). Our analysis results show that pre-training BERT on biomedical corpora helps it to understand complex biomedical texts. AVAILABILITY AND IMPLEMENTATION: We make the pre-trained weights of BioBERT freely available at https://github.com/naver/biobert-pretrained, and the source code for fine-tuning BioBERT available at https://github.com/dmis-lab/biobert.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.