ArticleJournal of speech, language, and hearing research : JSLHR2024
Toward Generalizable Machine Learning Models in Speech, Language, and Hearing Sciences: Estimating Sample Size and Reducing Overfitting.
Article in Journal of speech, language, and hearing research : JSLHR, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 30 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
30 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Machine learning models for predicting response to epidermal growth factor receptor tyrosine kinase inhibitors in non-small cell lung cancer brain metastases: a systematic review and meta-analysis.Clinical & translational oncology : official publication of the Federation of Spanish Oncology Societies and of the National Cancer Institute of Mexico · 2026Pooled it
- Pooled it
- The Effect of Phonotrauma Severity on the Discriminative Power of Different Ambulatory Voice Measures.Journal of speech, language, and hearing research : JSLHR · 2026Article
- Predicting anemia treatment outcomes in maintenance hemodialysis patients using multiple machine learning models.BMC medical informatics and decision making · 2026Article
- Participant-Aware Model Validation for Repeated-Measures Data: Comparative Cross-Validation Study.JMIR AI · 2026Article
- From Breath to Brain: NICU respiratory interventions and bedside brain signal entropy predict later autism risk.Developmental cognitive neuroscience · 2026Article
- Real-World Multimodal Machine Learning for Risk Enrichment Across the Alzheimer's Disease Spectrum.Journal of clinical medicine · 2026Article
- Zone of inhibition prediction in silver nanoparticle antimicrobial activity via physics-guided liquid state machine.Biomaterials and biosystems · 2026Article
- A community screening tool for neuropsychiatric symptoms in the elderly: integrating cortisol, microbiome, and social factors with machine learning.Translational psychiatry · 2026Observational
- Interpretable self-supervised transformers for resting-state EEG analysis in Alzheimer's disease.Frontiers in neuroinformatics · 2026Article
- Clinical risk factors and machine learning for venous thromboembolism in lung cancer: an exploratory single-center study.Frontiers in medicine · 2026Article
- Mitigating Limited Data Challenges to Improve Artificial Intelligence Integration in Rare Disease Drug Development.NEJM AI · 2025Article
- EEG biomarkers for Alzheimer's disease: A novel automated pipeline for detecting and monitoring disease progression.Journal of Alzheimer's disease : JAD · 2025Article
- A Comprehensive Review on the Application of Artificial Intelligence for Predicting Postsurgical Recurrence Risk in Early-Stage Non-Small Cell Lung Cancer Using Computed Tomography, Positron Emission Tomography, and Clinical Data.Journal of medical radiation sciences · 2025Review
- Comparative Evaluation of the Effectiveness and Efficiency of Computational Methods in the Detection of Asbestos Cement in Hyperspectral Images.Materials (Basel, Switzerland) · 2025Article
- Listening to the Mind: Integrating Vocal Biomarkers into Digital Health.Brain sciences · 2025Article
- Speech Acoustic Markers Can Detect Mild Cognitive Impairment in Parkinson's Disease.IEEE journal of selected topics in signal processing · 2025Article
- Predicting ICU Delirium in Critically Ill COVID-19 Patients Using Demographic, Clinical, and Laboratory Admission Data: A Machine Learning Approach.Life (Basel, Switzerland) · 2025Article
- Using ML techniques to predict extubation outcomes for patients with central nervous system injuries in the Yun-Gui Plateau.Scientific reports · 2025Article
- Automated Analysis of Relative Fundamental Frequency in Continuous Speech: Development and Comparison of Three Processing Pipelines.Journal of voice : official journal of the Voice Foundation · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
Abstract
purposeMany studies using machine learning (ML) in speech, language, and hearing sciences rely upon cross-validations with single data splitting. This study's first purpose is to provide quantitative evidence that would incentivize researchers to instead use the more robust data splitting method of nested
methodFirst, the significant impact of different cross-validations on ML outcomes was demonstrated using real-world clinical data. Then, Monte Carlo simulations were used to quantify the interactions among the employed cross-validation method, the discriminative power of features, the dimensionality of the feature space, the dimensionality of the model, and the sample size. Four different cross-validation methods (single holdout, 10-fold, train-validation-test, and nested 10-fold) were compared based on the statistical power and confidence of the resulting ML models. Distributions of the null and alternative hypotheses were used to determine the minimum required sample size for obtaining a statistically significant outcome (5% significance) with 80% power. Statistical confidence of the model was defined as the probability of correct features being selected for inclusion in the final model.
resultsML models generated based on the single holdout method had very low statistical power and confidence, leading to overestimation of classification accuracy. Conversely, the nested 10-fold cross-validation method resulted in the highest statistical confidence and power while also providing an unbiased estimate of accuracy. The required sample size using the single holdout method could be 50% higher than what would be needed if nested
conclusionThe adoption of nested SUPPLEMENTAL MATERIAL: https://doi.org/10.23641/asha.25237045.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.