Evidence map›Paper›PMID 36510134›Full record

SynthesisBMC medical research methodology2022

Methodological guidance for the evaluation and updating of clinical prediction models: a systematic review.

M A E Binuya, E G Engelhardt, W Schats, M K Schmidt, E W Steyerberg

Abstract readSystematic Review
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in BMC medical research methodology, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 138 papers, 11 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
138citing papers in PubMed, 11 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

138 citing papers in PubMed, 11 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Risk Prediction Models for Deep Vein Thrombosis in Patients With Spontaneous Intracerebral Hemorrhage: A Systematic Review and Meta-Analysis.Clinical and applied thrombosis/hemostasis : official journal of the International Academy of Clinical and Applied Thrombosis/Hemostasis
    Pooled it
  12. Trial
  13. Trial
  14. Article
  15. Article
  16. Article
  17. Article
  18. Review
  19. Article
  20. Article

78 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

5 authors.

M A E BinuyaDivision of Molecular Pathology, the Netherlands Cancer Institute - Antoni van Leeuwenhoek Hospital, Plesmanlaan 121, 1066 CX, Amsterdam, The Netherlands. m.binuya@nki.nl.
E G EngelhardtDivision of Molecular Pathology, the Netherlands Cancer Institute - Antoni van Leeuwenhoek Hospital, Plesmanlaan 121, 1066 CX, Amsterdam, The Netherlands.
W SchatsScientific Information Service, The Netherlands Cancer Institute - Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands.
M K SchmidtDivision of Molecular Pathology, the Netherlands Cancer Institute - Antoni van Leeuwenhoek Hospital, Plesmanlaan 121, 1066 CX, Amsterdam, The Netherlands.
E W SteyerbergDepartment of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundClinical prediction models are often not evaluated properly in specific settings or updated, for instance, with information from new markers. These key steps are needed such that models are fit for purpose and remain relevant in the long-term. We aimed to present an overview of methodological guidance for the evaluation (i.e., validation and impact assessment) and updating of clinical prediction models.

methodsWe systematically searched nine databases from January 2000 to January 2022 for articles in English with methodological recommendations for the post-derivation stages of interest. Qualitative analysis was used to summarize the 70 selected guidance papers.

resultsKey aspects for validation are the assessment of statistical performance using measures for discrimination (e.g., C-statistic) and calibration (e.g., calibration-in-the-large and calibration slope). For assessing impact or usefulness in clinical decision-making, recent papers advise using decision-analytic measures (e.g., the Net Benefit) over simplistic classification measures that ignore clinical consequences (e.g., accuracy, overall Net Reclassification Index). Commonly recommended methods for model updating are recalibration (i.e., adjustment of intercept or baseline hazard and/or slope), revision (i.e., re-estimation of individual predictor effects), and extension (i.e., addition of new markers). Additional methodological guidance is needed for newer types of updating (e.g., meta-model and dynamic updating) and machine learning-based models.

conclusionSubstantial guidance was found for model evaluation and more conventional updating of regression-based models. An important development in model evaluation is the introduction of a decision-analytic framework for assessing clinical usefulness. Consensus is emerging on methods for model updating.

Indexed as

Models, StatisticalCalibrationHumansPrognosisCalibrationClinical usefulnessDiscriminationImpact assessmentModel evaluationModel updatingPrediction modelValidation

Identifiers

PMID36510134
PMCPMC9742671

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.