ArticleBMJ (Clinical research ed.)2020
Consistency of variety of machine learning and statistical models in predicting clinical risks of individual patients: longitudinal cohort study using cardiovascular disease as exemplar.
Article in BMJ (Clinical research ed.), 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 87 papers, 6 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
87 citing papers in PubMed, 6 syntheses or guidelines pooled it.
- Comparing machine learning models and traditional approaches for predicting obstructive coronary artery disease: a systematic review and meta-analysis.Frontiers in cardiovascular medicine · 2026Pooled it
- Predictive models of Alzheimer's disease dementia risk in older adults with mild cognitive impairment: a systematic review and critical appraisal.BMC geriatrics · 2024Pooled it
- Artificial intelligence in the risk prediction models of cardiovascular disease and development of an independent validation screening tool: a systematic review.BMC medicine · 2024Pooled it
- Machine-learning versus traditional approaches for atherosclerotic cardiovascular risk prognostication in primary prevention cohorts: a systematic review and meta-analysis.European heart journal. Quality of care & clinical outcomes · 2023Pooled it
- Development and internal-external validation of statistical and machine learning models for breast cancer prognostication: cohort study.BMJ (Clinical research ed.) · 2023Pooled it
- Pooled it
- Explainable machine learning model reveals its decision-making process in identifying patients with paroxysmal atrial fibrillation at high risk for recurrence after catheter ablation.BMC cardiovascular disorders · 2023Trial
- Prediction of Acute Kidney Injury in Oncology: From Clinical Risk Scores to AI-Enabled Precision Onco-Nephrology.Cancers · 2026Review
- An interpretable machine learning model for early prediction of subsequent observed CRRT initiation in patients with sepsis-associated acute kidney injury.Clinical and experimental medicine · 2026Observational
- Deciphering the predictors of endometrial nonbenign lesions in asymptomatic postmenopausal women via explainable machine learning.Menopause (New York, N.Y.) · 2026Article
- Identification and Validation of an Explainable Predictive Model For Heart Failure in Patients With Hypertension.Reviews in cardiovascular medicine · 2026Article
- A feature selection-based oblique hyperplane for oblique random survival forests.BMC medical research methodology · 2026Article
- Prediction of medication overuse in patients with migraine using cox regression and machine learning: a real-world cohort.The journal of headache and pain · 2026Article
- Prediction models developed using artificial intelligence: similar predictive performances with highly varying predictions for individuals - an illustration in deep vein thrombosis.Diagnostic and prognostic research · 2026Article
- A transformer-based survival model for prediction of all-cause mortality in patients with heart failure: a multi-cohort study.NPJ digital medicine · 2026Article
- Early risk stratification of postoperative pneumonia after brain tumor surgery using routine perioperative variables: development and prospective multicenter validation of an interpretable prediction model.Frontiers in cellular and infection microbiology · 2026Article
- An Interpretable MRI-Based Machine Learning Model for Preoperative Identification of the Vascular Dissemination Phenotype in Hepatocellular Carcinoma: A Dual-Center Study.Journal of hepatocellular carcinoma · 2026Article
- Machine learning-derived predictive model for post-ERCP pancreatitis in patients with common bile duct stones: a retrospective multicenter study.Surgical endoscopy · 2025Article
- Machine learning-based management of hypertensive disorders in pregnancy: analysis of differences in key risk factors between gestational hypertension and pre-eclampsia and construction of a pre-eclampsia prediction model.European journal of medical research · 2025Article
- Development of an explainable machine learning model for 3-year cardiovascular risk prediction in new-onset type 2 diabetes using the TyG index and ultrasound features.BMC medical informatics and decision making · 2025Article
27 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveTo assess the consistency of machine learning and statistical techniques in predicting individual level and population level risks of cardiovascular disease and the effects of censoring on risk predictions.
designLongitudinal cohort study from 1 January 1998 to 31 December 2018. SETTING AND
participants3.6 million patients from the Clinical Practice Research Datalink registered at 391 general practices in England with linked hospital admission and mortality records.
main outcome measuresModel performance including discrimination, calibration, and consistency of individual risk prediction for the same patients among models with comparable model performance. 19 different prediction techniques were applied, including 12 families of machine learning models (grid searched for best models), three Cox proportional hazards models (local fitted, QRISK3, and Framingham), three parametric survival models, and one logistic model.
resultsThe various models had similar population level performance (C statistics of about 0.87 and similar calibration). However, the predictions for individual risks of cardiovascular disease varied widely between and within different types of machine learning and statistical models, especially in patients with higher risks. A patient with a risk of 9.5-10.5% predicted by QRISK3 had a risk of 2.9-9.2% in a random forest and 2.4-7.2% in a neural network. The differences in predicted risks between QRISK3 and a neural network ranged between -23.2% and 0.1% (95% range). Models that ignored censoring (that is, assumed censored patients to be event free) substantially underestimated risk of cardiovascular disease. Of the 223 815 patients with a cardiovascular disease risk above 7.5% with QRISK3, 57.8% would be reclassified below 7.5% when using another model.
conclusionsA variety of models predicted risks for the same patients very differently despite similar model performances. The logistic models and commonly used machine learning models should not be directly applied to the prediction of long term risks without considering censoring. Survival models that consider censoring and that are explainable, such as QRISK3, are preferable. The level of consistency within and between models should be routinely assessed before they are used for clinical decision making.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.