ArticleNPJ digital medicine2021
Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction.
Article in NPJ digital medicine, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 298 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
298 citing papers in PubMed, 5 syntheses or guidelines pooled it, 780 citations in OpenAlex.
- Pooled it
- Enhancing Patient Outcome Prediction Through Deep Learning With Sequential Diagnosis Codes From Structured Electronic Health Record Data: Systematic Review.Journal of medical Internet research · 2025Pooled it
- Review of applications of deep learning in veterinary diagnostics and animal health.Frontiers in veterinary science · 2025Pooled it
- Machine Learning Models for Pancreatic Cancer Risk Prediction Using Electronic Health Record Data-A Systematic Review and Assessment.The American journal of gastroenterology · 2024Pooled it
- Using natural language processing to analyze unstructured patient-reported outcomes data derived from electronic health records for cancer populations: a systematic review.Expert review of pharmacoeconomics & outcomes research · 2024Pooled it
- An LLM chatbot to facilitate primary-to-specialist care transitions: a randomized controlled trial.Nature medicine · 2026Trial
- Article
- CURENet: combining unified representations for efficient chronic disease prediction.Health information science and systems · 2026Article
- Article
- Large Language Models for Clinical Data Extraction in Workplace Injury Rehabilitation: Protocol for a Retrospective Pilot Study of Accuracy, Fairness, and Methodological Considerations in Workers' Compensation Medical Chart Review.JMIR research protocols · 2026Article
- Article
- Prediction of maternal and infant outcomes from longitudinal electronic health records with a mother-child AI agent.Nature medicine · 2026Article
- GERBEHRT: a BERT-based model tailored for German electronic health records - potential in chronic kidney disease prediction.BMC medical informatics and decision making · 2026Article
- Large Language Model-Based Clinical Decision Support for Antibiotic Selection and Dose Recommendation in Hospitalized Patients With Pneumonia: Multicenter Retrospective Study.JMIR medical informatics · 2026Article
- A vision-language pretrained transformer for versatile clinical respiratory disease applications.Nature biomedical engineering · 2026Article
- A meta-learning method with reduced data requirements for training and updating deep learning models on wearable devices.Communications medicine · 2026Article
- Transformers for molecular property prediction: domain adaptation efficiently improves performance.Journal of cheminformatics · 2026Article
- Uncertainty-calibrated adaptation of clinical transformer foundation models enhances in-hospital mortality and hospital readmission prediction.npj health systems · 2026Article
- Applying Artificial Intelligence and machine learning in precision nutrition.Nature communications · 2026Review
- Large language models are powerful electronic health record encoders.NPJ digital medicine · 2026Article
238 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors at 3 institutions in 2 countries.
Funding
Abstract
Deep learning (DL)-based predictive models from electronic health records (EHRs) deliver impressive performance in many clinical tasks. Large training cohorts, however, are often required by these models to achieve high accuracy, hindering the adoption of DL-based models in scenarios with limited training data. Recently, bidirectional encoder representations from transformers (BERT) and related models have achieved tremendous successes in the natural language processing domain. The pretraining of BERT on a very large training corpus generates contextualized embeddings that can boost the performance of models trained on smaller datasets. Inspired by BERT, we propose Med-BERT, which adapts the BERT framework originally developed for the text domain to the structured EHR domain. Med-BERT is a contextualized embedding model pretrained on a structured EHR dataset of 28,490,650 patients. Fine-tuning experiments showed that Med-BERT substantially improves the prediction accuracy, boosting the area under the receiver operating characteristics curve (AUC) by 1.21-6.14% in two disease prediction tasks from two clinical databases. In particular, pretrained Med-BERT obtains promising performances on tasks with small fine-tuning training sets and can boost the AUC by more than 20% or obtain an AUC as high as a model trained on a training set ten times larger, compared with deep learning models without Med-BERT. We believe that Med-BERT will benefit disease prediction studies with small local training datasets, reduce data collection expenses, and accelerate the pace of artificial intelligence aided healthcare.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.