ArticlePatterns (New York, N.Y.)2025
Unraveling overoptimism and publication bias in ML-driven science.
Article in Patterns (New York, N.Y.), 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 8 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
8 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Magnetic Resonance Imaging-Based Artificial Intelligence in Predicting Prostate Cancer Biochemical Recurrence: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2026Pooled it
- Artificial Intelligence for Predicting Postoperative Outcomes in Anterior Segment and Ocular Adnexa Surgery: A Systematic Review.Ophthalmology science · 2026Review
- A module-based approach for post-omics, post-GWAS network-based gene classification.Bioinformatics (Oxford, England) · 2026Article
- An End-User Audit of Reproducibility, Data Leakage, and Overfitting of the Top-Ranked ADMET Prediction Models in TDC Leaderboards.Journal of chemical information and modeling · 2026Article
- Machine learning-based models for tumor mutation burden prediction in gastrointestinal cancers: a systematic review and meta-analysis.Discover oncology · 2026Review
- Machine learning approaches for data-driven hydrocarbon bioaugmentation and phytoremediation: the role of multi-omics insights.Frontiers in microbiology · 2026Review
- Artificial Intelligence and Bipolar Disorder: Applications of Machine Learning Models for Diagnosis, Treatment, and Outcome Prediction.Alpha psychiatry · 2025Article
- Unraveling overoptimism and publication bias in ML-driven science.Patterns (New York, N.Y.) · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Machine learning (ML) is increasingly used across many disciplines with impressive reported results. However, recent studies suggest that the published performances of ML models are often overoptimistic. Validity concerns are underscored by findings of an inverse relationship between sample size and reported accuracy in published ML models, contrasting with the theory of learning curves where accuracy should improve or remain stable with increasing sample size. This paper investigates factors contributing to overoptimism in ML-driven science, focusing on overfitting and publication bias. We introduce a stochastic model for observed accuracy, integrating parametric learning curves and the aforementioned biases. We construct an estimator that corrects for these biases in observed data. Theoretical and empirical results show that our framework can estimate the underlying learning curve, providing realistic performance assessments from published results. By applying the model to meta-analyses of classifications of neurological conditions, we estimate the inherent limits of ML-driven prediction in each domain.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.