ArticleRadiology. Artificial intelligence2023
Generalizability of Machine Learning Models: Quantitative Evaluation of Three Methodological Pitfalls.
Article in Radiology. Artificial intelligence, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 60 papers, 6 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
60 citing papers in PubMed, 6 syntheses or guidelines pooled it, 115 citations in OpenAlex.
- Beyond the naked eye: a systematic review on the current state of radiomics approaches to the vestibular schwannoma.Journal of neuro-oncology · 2026Pooled it
- Machine Learning for Comparative Antidepressant Selection in Major Depressive Disorder: Systematic Review.JMIR mental health · 2026Pooled it
- Machine Learning in Left Ventricular Hypertrophy Detection: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2026Pooled it
- Performance of Machine Learning in Diagnosing KRAS (Kirsten Rat Sarcoma) Mutations in Colorectal Cancer: Systematic Review and Meta-Analysis.Journal of medical Internet research · 2025Pooled it
- Pooled it
- The use of artificial intelligence in musculoskeletal ultrasound: a systematic review of the literature.La Radiologia medica · 2024Pooled it
- Artificial intelligence and extracellular vesicles in oncology: towards tumor diagnosis, prediction, and therapy.Drug delivery · 2026Review
- Automated Assessment of OCT Angiography Image Quality Using the Artificial Intelligence Ready and Exploratory Atlas for Diabetes Insights Data Set.Ophthalmology science · 2026Article
- Explainable Stacked Ensemble Learning for Predicting Antibiotic Residues in the Danube River Within the Territory of the City of Novi Sad, Serbia.Antibiotics (Basel, Switzerland) · 2026Article
- SCRIPT: Stratified clinical risk prediction from pathology reports using large language models.Journal of pathology informatics · 2026Article
- Artificial Intelligence-Guided Optimization of Hyaluronic Acid-Coated Liposomal Linagliptin for Targeted Management of Polycystic Ovary Syndrome.AAPS PharmSciTech · 2026Article
- Advanced Technologies for the Diagnosis of Pulmonary Tuberculosis Using Exhaled Breath Samples: A Systematic Scoping Review.Tropical medicine & international health : TM & IH · 2026Article
- Radiomic analysis of medical imaging for classification in oncology and recommendations for clinically initiated research: a literature review.Quantitative imaging in medicine and surgery · 2026Review
- A physics-informed machine learning framework for predicting and mitigating doxorubicin nanocarrier toxicity in normal cells.Scientific reports · 2026Article
- A multi-task deep learning and radiomics framework for fetal anatomical structure detection and classification in ultrasound imaging.Scientific reports · 2026Article
- Inclusion of tumor periphery in radiomics analysis of magnetic resonance images does not improve predictions of preoperative therapy response in patients with rectal cancer.Abdominal radiology (New York) · 2026Article
- Comment on Iacobescu et al. Evaluating Binary Classifiers for Cardiovascular Disease Prediction: Enhancing Early Diagnostic Capabilities.Journal of cardiovascular development and disease · 2026Article
- Radiomics-based MRI model for differentiating ovarian cystadenoma and cystadenocarcinoma.Frontiers in oncology · 2026Article
- An interactive mindmap of blood film images for automated malaria diagnosis: a comprehensive metadata repository of available datasets.Bioinformatics advances · 2026Review
- Impact of ComBat harmonization on MRI radiomic-analysis of nasopharyngeal carcinoma: a dual-center study.Frontiers in oncology · 2026Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors at 3 institutions in 2 countries.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Purpose: To investigate the impact of the following three methodological pitfalls on model generalizability: ( Materials and Methods: The authors used retrospective CT, histopathologic analysis, and radiography datasets to develop machine learning models with and without the three methodological pitfalls to quantitatively illustrate their effect on model performance and generalizability. F1 score was used to measure performance, and differences in performance between models developed with and without errors were assessed using the Wilcoxon rank sum test when applicable. Results: Violation of the independence assumption by applying oversampling, feature selection, and data augmentation before splitting data into training, validation, and test sets seemingly improved model F1 scores by 71.2% for predicting local recurrence and 5.0% for predicting 3-year overall survival in head and neck cancer and by 46.0% for distinguishing histopathologic patterns in lung cancer. Randomly distributing data points for a patient across datasets superficially improved the F1 score by 21.8%. High model performance metrics did not indicate high-quality lung segmentation. In the presence of a batch effect, a model built for pneumonia detection had an F1 score of 98.7% but correctly classified only 3.86% of samples from a new dataset of healthy patients. Conclusion: Machine learning models developed with these methodological pitfalls, which are undetectable during internal evaluation, produce inaccurate predictions; thus, understanding and avoiding these pitfalls is necessary for developing generalizable models.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.