ArticleBMC medicine2023
Clinical prediction models and the multiverse of madness.
Article in BMC medicine, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 35 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
35 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- Prognostic models in populations with heart failure: a systematic review and meta-analysis.Systematic reviews · 2026Pooled it
- 2024 Revision of the level of evidence grading system for ESC clinical practice guideline recommendations II: diagnostic tests and prediction models.European heart journal · 2025Guideline
- Diagnostic models for sepsis-associated encephalopathy: a comprehensive systematic review and meta-analysis.Frontiers in neurology · 2025Pooled it
- MRI-based radiomics prediction of somatostatin receptor ligand response in acromegaly: an automated pipeline study.Journal of the Endocrine Society · 2026Article
- Big data and psychiatry: advances, constraints and future directions.World psychiatry : official journal of the World Psychiatric Association (WPA) · 2026Article
- Comparison of Methods for Incorporating Related Data When Developing Clinical Prediction Models: A Simulation Study.Biometrical journal. Biometrische Zeitschrift · 2026Article
- Stratified Analysis of Patients Within the PSA Gray Zone (4-10 ng/mL) and Its Clinical Application Value: Development of a Predictive Model for Clinically Significant Prostate Cancer Using Quantitative Indicators of PSA, ADC, and Relative T2 Value.Diagnostics (Basel, Switzerland) · 2026Article
- Development and evaluation of the electronic frailty index+ (eFI+) tool for older people: prognostic prediction modelling with integrated decision curve and health economic analysis.Health technology assessment (Winchester, England) · 2026Observational
- Beyond Area Under the Receiver Operating Characteristic Curve: Evaluating Predictive Performance Metrics Under Class Imbalance in Real-World Clinical Data.JMIR formative research · 2026Article
- A decomposition of Fisher's information to inform sample size for developing or updating fair and precise clinical prediction models - part 3: continuous outcomes.Diagnostic and prognostic research · 2026Article
- Risk Prediction in Cardio-Oncology: Conceptual and Methodological Considerations: JACC: CardioOncology State-of-the-Art Review.JACC. CardioOncology · 2026Review
- An international multi-centre study to develop and validate federated learning-based prognostic models for anal cancer.Nature communications · 2026Article
- Innovations to improve outcomes and uptake of psychotherapies for mental disorders: a state-of-the-art review.World psychiatry : official journal of the World Psychiatric Association (WPA) · 2026Article
- External validation of three risk prediction models for deep vein thrombosis in patients with acute stroke: a single-center cohort study.Frontiers in cardiovascular medicine · 2026Article
- Influence of Continuous Predictor Modelling Methods on Prediction Stability in Clinical Prediction Model Development: An Empirical Comparison Using Real Clinical Data.Clinical epidemiology · 2026Article
- A seven-variable clinical prediction model for bone metastasis at initial diagnosis of prostate cancer.American journal of cancer research · 2026Article
- Association between Three-dimensional Margin Assessment and Local Tumor Control in Percutaneous Lung Cryoablation of Colorectal Metastases.Radiology. Imaging cancer · 2026Article
- Beyond Comparing Machine Learning and Logistic Regression in Clinical Prediction Modelling: Shifting from Model Debate to Data Quality.Journal of medical Internet research · 2025Article
- Construction and validation of a multi-dimensional health indicator-driven osteoporosis risk prediction model: a large-sample cross-sectional study based on two centers.BMC musculoskeletal disorders · 2025Article
- MRDadaptis: self-adaptive parameter configuration enhances minimal residual disease detection in heterogeneous ctDNA samples.Briefings in bioinformatics · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
Abstract
backgroundEach year, thousands of clinical prediction models are developed to make predictions (e.g. estimated risk) to inform individual diagnosis and prognosis in healthcare. However, most are not reliable for use in clinical practice. MAIN BODY: We discuss how the creation of a prediction model (e.g. using regression or machine learning methods) is dependent on the sample and size of data used to develop it-were a different sample of the same size used from the same overarching population, the developed model could be very different even when the same model development methods are used. In other words, for each model created, there exists a multiverse of other potential models for that sample size and, crucially, an individual's predicted value (e.g. estimated risk) may vary greatly across this multiverse. The more an individual's prediction varies across the multiverse, the greater the instability. We show how small development datasets lead to more different models in the multiverse, often with vastly unstable individual predictions, and explain how this can be exposed by using bootstrapping and presenting instability plots. We recommend healthcare researchers seek to use large model development datasets to reduce instability concerns. This is especially important to ensure reliability across subgroups and improve model fairness in practice.
conclusionsInstability is concerning as an individual's predicted value is used to guide their counselling, resource prioritisation, and clinical decision making. If different samples lead to different models with very different predictions for the same individual, then this should cast doubt into using a particular model for that individual. Therefore, visualising, quantifying and reporting the instability in individual-level predictions is essential when proposing a new model.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.