ArticleScientific data2022
Peeking into a black box, the fairness and generalizability of a MIMIC-III benchmarking model.
Article in Scientific data, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 35 papers, 4 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
35 citing papers in PubMed, 4 syntheses or guidelines pooled it, 62 citations in OpenAlex.
- Explainability, Bias and Generalizability of AI Models in Dentistry: A Systematic Review of Model Interpretability and Equity.Clinical and experimental dental research · 2026Pooled it
- Artificial intelligence models in the surgical planning of low-grade gliomas: a systematic review.Frontiers in oncology · 2025Pooled it
- Unmasking bias in artificial intelligence: a systematic review of bias detection and mitigation strategies in electronic health record-based models.Journal of the American Medical Informatics Association : JAMIA · 2024Pooled it
- Predict, diagnose, and treat chronic kidney disease with machine learning: a systematic literature review.Journal of nephrology · 2023Pooled it
- Performance and Fairness of a Peripheral Artery Disease Detection Model Across Patient Phenotypes and Health Systems.medRxiv : the preprint server for health sciences · 2026Article
- Critical appraisal of fairness metrics for artificial intelligence-based clinical prediction models: a scoping review.The Lancet. Digital health · 2026Article
- Advancing health equity in proactive health management: from data underrepresentation and algorithmic bias to a closed-loop governance framework.Frontiers in public health · 2026Review
- Peptide-Based Therapeutics for Alzheimer's Disease: Medicinal Chemistry, AI-Guided Computational Design, and Blood-Brain Barrier Delivery.Drug design, development and therapy · 2026Review
- Evaluating the impact of data biases on algorithmic fairness and clinical utility of machine learning models for prolonged opioid use prediction.JAMIA open · 2025Article
- What Is Fair? Defining Fairness in Machine Learning for Health.Statistics in medicine · 2025Review
- Scientific Evidence for Clinical Text Summarization Using Large Language Models: Scoping Review.Journal of medical Internet research · 2025Article
- A scoping review of machine learning models to predict risk of falls in elders, without using sensor data.Diagnostic and prognostic research · 2025Review
- Clinical trials informed framework for real world clinical implementation and deployment of artificial intelligence applications.NPJ digital medicine · 2025Review
- Improving medical machine learning models with generative balancing for equity and excellence.NPJ digital medicine · 2025Article
- Global Research Landscape of Artificial Intelligence in Urology: A Systematic Analysis of Emerging Trends, Clinical Impact, and Collaborative Networks (1971-2024).Medical journal of the Islamic Republic of Iran · 2025Article
- Applying Large Language Models to Assess Quality of Care: Monitoring ADHD Medication Side Effects.Pediatrics · 2025Article
- Bias in medical AI: Implications for clinical decision-making.PLOS digital health · 2024Review
- A survey of recent methods for addressing AI fairness and bias in biomedicine.Journal of biomedical informatics · 2024Review
- Leveraging a Large Language Model to Assess Quality-of-Care: Monitoring ADHD Medication Side Effects.medRxiv : the preprint server for health sciences · 2024Article
- Towards global model generalizability: independent cross-site feature evaluation for patient-level risk prediction models using the OHDSI network.Journal of the American Medical Informatics Association : JAMIA · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors at 3 institutions in 2 countries.
Funding
Abstract
As artificial intelligence (AI) makes continuous progress to improve quality of care for some patients by leveraging ever increasing amounts of digital health data, others are left behind. Empirical evaluation studies are required to keep biased AI models from reinforcing systemic health disparities faced by minority populations through dangerous feedback loops. The aim of this study is to raise broad awareness of the pervasive challenges around bias and fairness in risk prediction models. We performed a case study on a MIMIC-trained benchmarking model using a broadly applicable fairness and generalizability assessment framework. While open-science benchmarks are crucial to overcome many study limitations today, this case study revealed a strong class imbalance problem as well as fairness concerns for Black and publicly insured ICU patients. Therefore, we advocate for the widespread use of comprehensive fairness and performance assessment frameworks to effectively monitor and validate benchmark pipelines built on open data resources.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.