ReviewBMJ open2019
Empirical evidence of the impact of study characteristics on the performance of prediction models: a meta-epidemiological study.
Review in BMJ open, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
17 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- Comprehensive comparison of stroke risk score performance: a systematic review and meta-analysis among 6 267 728 patients with atrial fibrillation.Europace : European pacing, arrhythmias, and cardiac electrophysiology : journal of the working groups on cardiac pacing, arrhythmias, and cardiac cellular electrophysiology of the European Society of Cardiology · 2022Pooled it
- Trial-level characteristics associate with treatment effect estimates: a systematic review of meta-epidemiological studies.BMC medical research methodology · 2022Pooled it
- Performance of the Framingham risk models and pooled cohort equations for predicting 10-year risk of cardiovascular disease: a systematic review and meta-analysis.BMC medicine · 2019Pooled it
- Prediction Models for Periprosthetic Joint Infection: A Systematic Review of Traditional and Machine Learning Approaches.Mayo Clinic proceedings. Digital health · 2026Review
- Application of a Pediatric Tracheostomy-Specific Risk Tier System Using Administrative Data.JAMA otolaryngology-- head & neck surgery · 2026Article
- Comprehensive analysis of predictive models for disease manifestations and case fatality in systemic lupus erythematosus.NPJ digital medicine · 2026Review
- Predicting childhood overweight and obesity at school entrance using healthcare, demographic, and socioeconomic data in Wales, UK.European journal of public health · 2026Article
- Subchondral bone alterations in temporomandibulat joint arthralgia negatively affect the functional results of stabilization appliance therapy. A retrospective exploratory cohort study.Scientific reports · 2026Article
- Clinical factors associated with radiographic severity of progressive temporomandibular joint osteoarthritis: a retrospective CBCT study.BMC musculoskeletal disorders · 2025Article
- Factors associated with voice-related quality of life among patients with temporomandibular disorders.Journal of applied oral science : revista FOB · 2024Article
- Comparative analysis of CRASH and IMPACT in predicting the outcome of 340 patients with traumatic brain injury.Translational neuroscience · 2024Article
- Recurrent venous thromboembolism and bleeding with extended anticoagulation: the VTE-PREDICT risk score.European heart journal · 2023Article
- Early identification of bleeding in trauma patients: external validation of traumatic bleeding scores in the Swiss Trauma Registry.Critical care (London, England) · 2022Article
- A Unified Framework on Generalizability of Clinical Prediction Models.Frontiers in artificial intelligence · 2022Review
- A newly developed and externally validated non-clinical score accurately predicts 10-year cardiovascular disease risk in the general adult population.Scientific reports · 2021Article
- A logistic analysis prediction model of TMJ condylar erosion in patients with TMJ arthralgia.BMC oral health · 2021Article
- Validation of the BATT score for prehospital risk stratification of traumatic haemorrhagic death: usefulness for tranexamic acid treatment criteria.Scandinavian journal of trauma, resuscitation and emergency medicine · 2021Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectivesTo empirically assess the relation between study characteristics and prognostic model performance in external validation studies of multivariable prognostic models.
designMeta-epidemiological study. DATA SOURCES AND STUDY SELECTION: On 16 October 2018, we searched electronic databases for systematic reviews of prognostic models. Reviews from non-overlapping clinical fields were selected if they reported common performance measures (either the concordance (c)-statistic or the ratio of observed over expected number of events (OE ratio)) from 10 or more validations of the same prognostic model. DATA EXTRACTION AND ANALYSES: Study design features, population characteristics, methods of predictor and outcome assessment, and the aforementioned performance measures were extracted from the included external validation studies. Random effects meta-regression was used to quantify the association between the study characteristics and model performance.
resultsWe included 10 systematic reviews, describing a total of 224 external validations, of which 221 reported c-statistics and 124 OE ratios. Associations between study characteristics and model performance were heterogeneous across systematic reviews. C-statistics were most associated with variation in population characteristics, outcome definitions and measurement and predictor substitution. For example, validations with eligibility criteria comparable to the development study were associated with higher c-statistics compared with narrower criteria (difference in logit c-statistic 0.21(95% CI 0.07 to 0.35), similar to an increase from 0.70 to 0.74). Using a case-control design was associated with higher OE ratios, compared with using data from a cohort (difference in log OE ratio 0.97(95% CI 0.38 to 1.55), similar to an increase in OE ratio from 1.00 to 2.63).
conclusionsVariation in performance of prognostic models across studies is mainly associated with variation in case-mix, study designs, outcome definitions and measurement methods and predictor substitution. Researchers developing and validating prognostic models should realise the potential influence of these study characteristics on the predictive performance of prognostic models.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.