ArticleJournal of biomedical informatics2020
Detection of calibration drift in clinical prediction models to inform model updating.
Article in Journal of biomedical informatics, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 90 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
90 citing papers in PubMed, 5 syntheses or guidelines pooled it.
- Accuracy of Machine Learning Algorithms Based on Electroencephalogram in Sleep Apnea Detection: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2026Pooled it
- A Systematic Review and Meta-analysis of Externally Validated Epic Clinical Decision Support Tools.Journal of general internal medicine · 2026Pooled it
- Prostate cancer research on social media platforms: a bibliometric and thematic analysis.Frontiers in oncology · 2026Pooled it
- Prediction of frailty in community older adults based on machine learning: a systematic review and meta-analysis.Frontiers in public health · 2025Pooled it
- A survey on detecting healthcare concept drift in AI/ML models from a finance perspective.Frontiers in artificial intelligence · 2022Pooled it
- Evaluation of domain generalization and adaptation on improving model robustness to temporal dataset shift in clinical medicine.Scientific reports · 2022Trial
- Artificial intelligence in cardiovascular medicine: a practical guide for clinicians and researchers.European heart journal. Digital health · 2026Review
- Beyond Regulatory Approval: Lifecycle Assurance for Clinical Artificial Intelligence.Journal of medical systems · 2026Article
- Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradation.IEEE transactions on bio-medical engineering · 2026Review
- Machine Learning on the Pediatric Intensive Care (PIC) Database: PIC-Powered Prediction.Bioengineering (Basel, Switzerland) · 2026Review
- Detecting Adverse Events Following Acute Care Encounters among Veterans Prescribed Non-Steroidal Anti-Inflammatory Drugs.Journal of the American College of Emergency Physicians open · 2026Article
- Case-mix-adjusted benchmarking of long-term breast cancer-specific mortality: Insights from a nationwide Swedish registry.Breast (Edinburgh, Scotland) · 2026Article
- Bias and Fairness Across the Healthcare AI Lifecycle: A Clinician-Oriented Review.Balkan medical journal · 2026Review
- The HALO Model: A Learning Health System Framework for Artificial Intelligence.Learning health systems · 2026Article
- Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems.PLOS digital health · 2026Article
- FIRST-ICU: forecasting interventions and risk stratification in the ICU using graph neural network autoencoders.NPJ digital medicine · 2026Article
- A real-world feasibility evaluation of LLM-based clinical prediction: emergency department return visit admission across two academic medical centers.Research square · 2026Article
- Prospective validation and real-time implementation of an automated machine learning postoperative mortality prediction model.British journal of anaesthesia · 2026Article
- Machine learning-based risk prediction of 28-day mortality for sepsis patients with augmented renal clearance.Scientific reports · 2026Article
- Risk prediction in spine surgery: a scoping review of traditional models, artificial intelligence, and the challenge of clinical translation.Spine deformity · 2026Review
30 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
Abstract
Model calibration, critical to the success and safety of clinical prediction models, deteriorates over time in response to the dynamic nature of clinical environments. To support informed, data-driven model updating strategies, we present and evaluate a calibration drift detection system. Methods are developed for maintaining dynamic calibration curves with optimized online stochastic gradient descent and for detecting increasing miscalibration with adaptive sliding windows. These methods are generalizable to support diverse prediction models developed using a variety of learning algorithms and customizable to address the unique needs of clinical use cases. In both simulation and case studies, our system accurately detected calibration drift. When drift is detected, our system further provides actionable alerts by including information on a window of recent data that may be appropriate for model updating. Simulations showed these windows were primarily composed of data accruing after drift onset, supporting the potential utility of the windows for model updating. By promoting model updating as calibration deteriorates rather than on pre-determined schedules, implementations of our drift detection system may minimize interim periods of insufficient model accuracy and focus analytic resources on those models most in need of attention.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.