ArticleJournal of the American Statistical Association2024
Cross-validation: what does it estimate and how well does it do it?
Article in Journal of the American Statistical Association, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 129 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
129 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Risk and Protective Factors Associated With HIV-Related Mortality in Children Receiving Antiretroviral Therapy: A Model-Based Meta-Regression.JAMA pediatrics · 2026Pooled it
- Development and validation of a machine learning-based prediction model for prolonged length of stay after laparoscopic gastrointestinal surgery: a secondary analysis of the FDP-PONV trial.BMC gastroenterology · 2025Trial
- Individual patient disease activity expression profiles modify the relation between disease activity and structural damage progression in RA.EULAR rheumatology open · 2026Article
- Periodontitis Prediction Model Using Linked Electronic Health and Dental Records.JDR clinical and translational research · 2026Article
- Utilizing Machine Learning to Identify Multimodal Signatures for Patients Who Would Benefit from the Addition of Tremelimumab to Durvalumab and Chemotherapy (TRIDENT).Clinical cancer research : an official journal of the American Association for Cancer Research · 2026Observational
- Derivation and Validation of Charlson, Elixhauser and RxRisk Cost-Based Comorbidity Indices: A Register-Based Study of 350,000 Individuals Over 11 Years.PharmacoEconomics · 2026Article
- Prediction of Clinically Meaningful Improvement After Internet-Delivered Cognitive Behavioral Therapy for Depression and Anxiety Disorders: Machine Learning-Based Predictive Model Development and Temporal Validation Study.Journal of medical Internet research · 2026Article
- Context-Aware Multilevel Classification of Semantic Relations in Drug-Adverse Drug Reaction (ADR) Networks-Predicting Drug-Induced Liver Injury (DILI) as a Case Study.Chemical research in toxicology · 2026Article
- Soybean aphid (Hemiptera: Aphididae) population dynamics are associated with the temporal scale and pattern of soil moisture in soybean fields.Journal of insect science (Online) · 2026Article
- The functional neurobiology of dispositions towards negative emotions.Nature communications · 2026Article
- A Systematic Survey and Benchmark of Deep Learning for Molecular Property Prediction in the Foundation Model Era.Journal of chemical theory and computation · 2026Review
- Widespread use of invalid statistical tests in biomedical machine learning.bioRxiv : the preprint server for biology · 2026Article
- A Novel Radiomics-Integrated Panel for Preoperative Stratification of Pancreatic Neuroendocrine Tumors (PNETs).Cancers · 2026Article
- Compressive strength of mortar incorporating recycled nonmetallic printed circuit board waste: experimental and interpretable machine learning analysis.Scientific reports · 2026Article
- An investigation into in-sample and out-of-sample model selection for nonstationary autoregressive models.The British journal of mathematical and statistical psychology · 2026Article
- MRI Deep Learning for Differentiating Glioblastoma, IDH Wild-type from Central Nervous System Diffuse Large B-cell Lymphoma.Cancer research communications · 2026Article
- Multimodal Integration of Ambulatory ECG and Clinical Features for Sudden Cardiac Death and Pump Failure Death PredictionmedRxiv : the preprint server for health sciences · 2026Article
- Machine learning prediction of metabolic dysregulation in women with polycystic ovary syndrome: development and validation of hemato-inflammatory predictive models.BMC endocrine disorders · 2026Article
- Digital Monitoring of Pre-Exposure Prophylaxis Users Through Electronic Patient-Reported Outcome Measures and Electronic Patient-Reported Experience Measures: Multicenter Prospective Study on Feasibility, Safety, and Predictive Modeling of Digital Engagement.Journal of medical Internet research · 2026Observational
- Antigen heterogeneity in the development and clinical validation of a multiplexed urine test for tuberculosis.Communications medicine · 2026Article
69 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
Abstract
Cross-validation is a widely-used technique to estimate prediction error, but its behavior is complex and not fully understood. Ideally, one would like to think that cross-validation estimates the prediction error for the model at hand, fit to the training data. We prove that this is not the case for the linear model fit by ordinary least squares; rather it estimates the average prediction error of models fit on other unseen training sets drawn from the same population. We further show that this phenomenon occurs for most popular estimates of prediction error, including data splitting, bootstrapping, and Mallow's
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.