ArticleScientific reports2023
EHR foundation models improve robustness in the presence of temporal distribution shift.
Article in Scientific reports, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 31 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
31 citing papers in PubMed, 2 syntheses or guidelines pooled it, 44 citations in OpenAlex.
- Enhancing Patient Outcome Prediction Through Deep Learning With Sequential Diagnosis Codes From Structured Electronic Health Record Data: Systematic Review.Journal of medical Internet research · 2025Pooled it
- Machine learning techniques for predicting neurodevelopmental impairments in premature infants: a systematic review.Frontiers in artificial intelligence · 2025Pooled it
- Toward scalable early cancer detection: evaluating EHR-based predictive models against traditional screening criteria.NPJ precision oncology · 2026Article
- SurvivEHR: a competing risks, time-to-event foundation model for multiple long-term conditions from primary care electronic health records.NPJ digital medicine · 2026Article
- Fair and robust early readmission risk prediction from electronic health records via diffusion-based data augmentation and causal-invariant representation learning.Scientific reports · 2026Article
- The portability paradox of foundation models for clinical decision support.NPJ digital medicine · 2026Article
- Prediction of antibiotic-associated cutaneous adverse drug reactions using electronic health record foundation models.NPJ digital medicine · 2026Article
- Article
- Knowledgeable Language Models as Black-Box Optimizers for Personalized Medicine.... International Conference on Learning Representations · 2026Article
- Large Models for Small Tables: Adapting Tabular Foundation Models to EHR Data.AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science · 2026Article
- Clinician-in-the-loop screening saturation: predicting annotation yield for efficient EHR review.BMC medical informatics and decision making · 2025Article
- Current Perspectives on the Artificial Intelligence in Critical Care Medicine.Anesthesiology clinics · 2025Review
- A scoping review of self-supervised representation learning for clinical decision making using EHR categorical data.NPJ digital medicine · 2025Article
- Trajectory-Ordered Objectives for Self-Supervised Representation Learning of Temporal Healthcare Data Using Transformers: Model Development and Evaluation Study.JMIR medical informatics · 2025Article
- Rethinking model prototyping through the MedMNIST+ dataset collection.Scientific reports · 2025Article
- DOME: Directional medical embedding vectors from Electronic Health Records.Journal of biomedical informatics · 2025Article
- A roadmap to implementing machine learning in healthcare: from concept to practice.Frontiers in digital health · 2025Article
- Adaptive noise-augmented attention for enhancing Transformer fine-tuning on longitudinal medical data.Frontiers in artificial intelligence · 2025Article
- Reimagining clinical AI: from clickstreams to clinical insights with EHR use metadata.npj health systems · 2025Review
- From theoretical models to practical deployment: A perspective and case study of opportunities and challenges in AI-driven cardiac auscultation research for low-income settings.PLOS digital health · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
9 authors at 5 institutions in 3 countries.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Temporal distribution shift negatively impacts the performance of clinical prediction models over time. Pretraining foundation models using self-supervised learning on electronic health records (EHR) may be effective in acquiring informative global patterns that can improve the robustness of task-specific models. The objective was to evaluate the utility of EHR foundation models in improving the in-distribution (ID) and out-of-distribution (OOD) performance of clinical prediction models. Transformer- and gated recurrent unit-based foundation models were pretrained on EHR of up to 1.8 M patients (382 M coded events) collected within pre-determined year groups (e.g., 2009-2012) and were subsequently used to construct patient representations for patients admitted to inpatient units. These representations were used to train logistic regression models to predict hospital mortality, long length of stay, 30-day readmission, and ICU admission. We compared our EHR foundation models with baseline logistic regression models learned on count-based representations (count-LR) in ID and OOD year groups. Performance was measured using area-under-the-receiver-operating-characteristic curve (AUROC), area-under-the-precision-recall curve, and absolute calibration error. Both transformer and recurrent-based foundation models generally showed better ID and OOD discrimination relative to count-LR and often exhibited less decay in tasks where there is observable degradation of discrimination performance (average AUROC decay of 3% for transformer-based foundation model vs. 7% for count-LR after 5-9 years). In addition, the performance and robustness of transformer-based foundation models continued to improve as pretraining set size increased. These results suggest that pretraining EHR foundation models at scale is a useful approach for developing clinical prediction models that perform well in the presence of temporal distribution shift.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.