ArticleGigaScience2025
External validation of machine learning models-registered models and adaptive sample splitting.
Article in GigaScience, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 20 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
20 citing papers in PubMed, 5 syntheses or guidelines pooled it.
- Artificial Intelligence Diagnosis of Obstructive Sleep Apnea Using Overnight Pulse Oximetry: A Systematic Review and Bayesian Meta-Analysis.Journal of medical Internet research · 2026Pooled it
- The Role of Machine Learning in Talent Identification for Team Sports: A Systematic Review.Journal of sports science & medicine · 2026Pooled it
- Machine Learning for Predicting Stroke Risk Stratification Using Multiomics Data: Systematic Review.Journal of medical Internet research · 2026Pooled it
- Machine learning-based prediction models for noninvasive respiratory support failure in acute respiratory failure: a systematic review and meta-analysis.Frontiers in medicine · 2026Pooled it
- Prediction models for the occurrence and mortality of sepsis-associated lung injury: a systematic review and meta-analysis.Frontiers in medicine · 2026Pooled it
- Beyond predictive accuracy: A case for mechanism-informed, uncertainty-aware machine learning in food microbiology.iScience · 2026Review
- Current research landscape and future prospects of in silico modeling approaches for atopic dermatitis.JID innovations : skin science from molecules to population health · 2026Review
- Artificial intelligence and machine learning in heart and lung transplantation.EClinicalMedicine · 2026Review
- Machine-Learning-Driven Optimization of Functional Excipients and Their Biointeractions in Drug Formulations.ACS pharmacology & translational science · 2026Review
- Predicting End-Stage Renal Disease and Mortality in Chronic Kidney Disease Using Machine Learning: Retrospective Cohort Study.JMIR medical informatics · 2026Article
- Liposome Particle Size Prediction by In-Line Process Analytical Technology (PAT)-Integrated Machine Learning.Small methods · 2026Article
- Data-Driven Inverse Design of Silver Nanoparticle Size for Controlled Synthesis Across Multiple Systems Using Conditional Generative Models.Materials (Basel, Switzerland) · 2026Article
- TRIAGE: Trustworthy Reporting and Assessment for Clinical Gain and Effectiveness of AI Models.Diagnostics (Basel, Switzerland) · 2026Review
- Sample size requirements for machine learning classification of binary outcomes in bulk RNA-Seq data.BMC bioinformatics · 2026Article
- A multicenter, clinically interpretable prediction model for malignancy risk in C-TIRADS 3-4 thyroid nodules.Frontiers in oncology · 2026Article
- Childhood asthma and the microbiome: from gut-lung axis mechanisms to precision prevention strategies.Frontiers in immunology · 2026Review
- Leveraging Machine Learning to Advance Alcohol Research: Current Applications, Challenges, and Opportunities.Alcohol research : current reviews · 2026Review
- Review
- External validation of machine learning models-registered models and adaptive sample splitting.GigaScience · 2025Article
- The Rise of Artificial Intelligence in Orthopedics: A Bibliometric and Visualization Analysis.Journal of multidisciplinary healthcare · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
Abstract
backgroundMultivariate predictive models play a crucial role in enhancing our understanding of complex biological systems and in developing innovative, replicable tools for translational medical research. However, the complexity of machine learning methods and extensive data preprocessing and feature engineering pipelines can lead to overfitting and poor generalizability. An unbiased evaluation of predictive models necessitates external validation, which involves testing the finalized model on independent data. Despite its importance, external validation is often neglected in practice due to the associated costs.
resultsHere we propose that, for maximal credibility, model discovery and external validation should be separated by the public disclosure (e.g., preregistration) of feature processing steps and model weights. Furthermore, we introduce a novel approach to optimize the trade-off between efforts spent on model discovery and external validation in such studies. We show on data involving more than 3,000 participants from four different datasets that, for any "sample size budget," the proposed adaptive splitting approach can successfully identify the optimal time to stop model discovery so that predictive performance is maximized without risking a low-powered, and thus inconclusive, external validation.
conclusionThe proposed design and splitting approach (implemented in the Python package "AdaptiveSplit") may contribute to addressing issues of replicability, effect size inflation, and generalizability in predictive modeling studies.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.