ArticleBMC bioinformatics2022
Investigation of improving the pre-training and fine-tuning of BERT model for biomedical relation extraction.
Article in BMC bioinformatics, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
17 citing papers in PubMed.
- A Novel BERT-Based Machine Learning Approach for Enhanced CSF Leak Prediction in Endoscopic Endonasal Skull Base Surgery.Journal of neurological surgery. Part B, Skull base · 2026Article
- Dual channel drug-drug interactions extraction based on cross attention.BMC bioinformatics · 2026Article
- Unifying pJournal of chemical information and modeling · 2026Article
- A software pipeline for medical information extraction with large language models, open source and suitable for oncology.NPJ precision oncology · 2025Review
- An Informatics Approach to Characterizing Rarely Documented Clinical Information in Electronic Health Records: Spiritual Care as an Exemplar.Applied clinical informatics · 2025Article
- Predicting 30-Day Postoperative Mortality and American Society of Anesthesiologists Physical Status Using Retrieval-Augmented Large Language Models: Development and Validation Study.Journal of medical Internet research · 2025Article
- GlycoSiteMiner: an ML/AI-assisted literature mining-based pipeline for extracting glycosylation sites from PubMed abstracts.Glycobiology · 2025Article
- Iterative refinement and goal articulation to optimize large language models for clinical information extraction.NPJ digital medicine · 2025Article
- Prompts to Table: Specification and Iterative Refinement for Clinical Information Extraction with Large Language Models.medRxiv : the preprint server for health sciences · 2025Article
- Prospects and perils of ChatGPT in diabetes.World journal of diabetes · 2025Article
- Biomedical relation extraction method based on ensemble learning and attention mechanism.BMC bioinformatics · 2024Article
- LLM-AIx: An open source pipeline for Information Extraction from unstructured medical text based on privacy preserving Large Language Models.medRxiv : the preprint server for health sciences · 2024Article
- Automatic extraction of transcriptional regulatory interactions of bacteria from biomedical literature using a BERT-based approach.Database : the journal of biological databases and curation · 2024Article
- A deep learning-driven discovery of berberine derivatives as novel antibacterial against multidrug-resistant Helicobacter pylori.Signal transduction and targeted therapy · 2024Article
- Sentiment Impact of Public Health Agency communication Strategies on TikTok under COVID-19 Normalization: Deep Learning Exploration.Zeitschrift fur Gesundheitswissenschaften = Journal of public health · 2023Article
- Precision information extraction for rare disease epidemiology at scale.Journal of translational medicine · 2023Article
- Leveraging Unlabeled Clinical Data to Boost Performance of Risk Stratification Models for Suspected Acute Coronary Syndrome.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
Abstract
backgroundRecently, automatically extracting biomedical relations has been a significant subject in biomedical research due to the rapid growth of biomedical literature. Since the adaptation to the biomedical domain, the transformer-based BERT models have produced leading results on many biomedical natural language processing tasks. In this work, we will explore the approaches to improve the BERT model for relation extraction tasks in both the pre-training and fine-tuning stages of its applications. In the pre-training stage, we add another level of BERT adaptation on sub-domain data to bridge the gap between domain knowledge and task-specific knowledge. Also, we propose methods to incorporate the ignored knowledge in the last layer of BERT to improve its fine-tuning.
resultsThe experiment results demonstrate that our approaches for pre-training and fine-tuning can improve the BERT model performance. After combining the two proposed techniques, our approach outperforms the original BERT models with averaged F1 score improvement of 2.1% on relation extraction tasks. Moreover, our approach achieves state-of-the-art performance on three relation extraction benchmark datasets.
conclusionsThe extra pre-training step on sub-domain data can help the BERT model generalization on specific tasks, and our proposed fine-tuning mechanism could utilize the knowledge in the last layer of BERT to boost the model performance. Furthermore, the combination of these two approaches further improves the performance of BERT model on the relation extraction tasks.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.