ArticleJournal of the American Medical Informatics Association : JAMIA2021
Addressing bias in prediction models by improving subpopulation calibration.
Article in Journal of the American Medical Informatics Association : JAMIA, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 29 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
29 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Assessment of Adherence to Reporting Guidelines by Commonly Used Clinical Prediction Models From a Single Vendor: A Systematic Review.JAMA network open · 2022Pooled it
- Integrating mathematical modelling and artificial intelligence to combat emerging viral syndemics: A systematic review.New microbes and new infections · 2026Review
- Longitudinal trajectories of hospitalizations in US older adults with heart failure.The journals of gerontology. Series A, Biological sciences and medical sciences · 2026Article
- Bias Mitigation Across the Healthcare Artificial Intelligence Lifecycle: A Structured Narrative Review.Cureus · 2026Review
- Strategies for mitigating artificial intelligence bias in healthcare: a systematic review.JAMIA open · 2026Review
- Review
- Performance of clinical prediction models for chronic kidney disease among people with diabetes: external validation using the Canadian Primary Care Sentinel Surveillance Network (CPCSSN).Diagnostic and prognostic research · 2025Article
- Regulating genome language models: navigating policy challenges at the intersection of AI and genetics.Human genetics · 2025Review
- FairFML: fair federated machine learning with a case study on reducing gender disparities in cardiac arrest outcome prediction.npj health systems · 2025Article
- Subpopulation-specific synthetic electronic health records can increase mortality prediction performance.JAMIA open · 2025Article
- Evaluating a predictive model of avoidable hospital events for race- and sex-based bias.Health services research · 2025Article
- Adding social determinants of health to the equation: Development of a cardiometabolic disease staging model using clinical and social determinants of health to predict type 2 diabetes.Diabetes, obesity & metabolism · 2025Article
- Emerging algorithmic bias: fairness drift as the next dimension of model maintenance and sustainability.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- A scoping review of fair machine learning techniques when using real-world data.Journal of biomedical informatics · 2024Article
- Trans-Balance: Reducing demographic disparity for prediction models in the presence of class imbalance.Journal of biomedical informatics · 2024Article
- Toward Identifying New Risk Aversions and Subsequent Limitations and Biases When Making De-identified Structured Data Sets Openly Available in a Post-LLM world.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2024Article
- Personalized event prediction for Electronic Health Records.Artificial intelligence in medicine · 2023Article
- Clinical utility gains from incorporating comorbidity and geographic location information into risk estimation equations for atherosclerotic cardiovascular disease.Journal of the American Medical Informatics Association : JAMIA · 2023Article
- Article
- Fair admission risk prediction with proportional multicalibration.Proceedings of machine learning research · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveTo illustrate the problem of subpopulation miscalibration, to adapt an algorithm for recalibration of the predictions, and to validate its performance. MATERIALS AND
methodsIn this retrospective cohort study, we evaluated the calibration of predictions based on the Pooled Cohort Equations (PCE) and the fracture risk assessment tool (FRAX) in the overall population and in subpopulations defined by the intersection of age, sex, ethnicity, socioeconomic status, and immigration history. We next applied the recalibration algorithm and assessed the change in calibration metrics, including calibration-in-the-large.
results1 021 041 patients were included in the PCE population, and 1 116 324 patients were included in the FRAX population. Baseline overall model calibration of the 2 tested models was good, but calibration in a substantial portion of the subpopulations was poor. After applying the algorithm, subpopulation calibration statistics were greatly improved, with the variance of the calibration-in-the-large values across all subpopulations reduced by 98.8% and 94.3% in the PCE and FRAX models, respectively. DISCUSSION: Prediction models in medicine are increasingly common. Calibration, the agreement between predicted and observed risks, is commonly poor for subpopulations that were underrepresented in the development set of the models, resulting in bias and reduced performance for these subpopulations. In this work, we empirically evaluated an adapted version of the fairness algorithm designed by Hebert-Johnson et al. (2017) and demonstrated its use in improving subpopulation miscalibration.
conclusionA postprocessing and model-independent fairness algorithm for recalibration of predictive models greatly decreases the bias of subpopulation miscalibration and thus increases fairness and equality.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.