Classifying voice disorders for machine learning: a pilot study using the USVAC-C2025 diagnostic framework.

Introduction: Machine learning for voice disorders relies heavily on accurate diagnostic classification, yet progress has been limited by inconsistent labelling and the absence of a reproducible framework suitable for clinical and computational use. This study aimed to develop and evaluate a multilayer classification system for voice disorder diagnosis tailored for machine learning applications, and to determine its inter- and intra-rater reliability among otolaryngologists and speech-language pathologists. Method: We conducted a diagnostic reliability study of 45 adults with voice disorders who underwent comprehensive clinical assessment, including videostroboscopy, at a tertiary voice clinic in Sydney, Australia, between February 2018 and March 2024. A multidisciplinary team developed a five-level hierarchical classification framework through iterative consensus. Four blinded raters independently applied the framework to anonymised video and clinical datasets, with 15 cases randomly repeated for intra-rater analysis. Reliability was quantified using Fleiss Results: Intra-rater reliability was high (intraclass correlation coefficient range, 0.768-0.865), with comparable consistency across disciplines. Inter-rater reliability was strongest for identifying disordered vs. non-disordered voices ( Conclusion: These findings show that a structured, multilayer framework improves diagnostic consistency where machine learning systems most rely on stable labels and highlights key areas of diagnostic ambiguity. The system provides a practical foundation for creating reliable annotated datasets and supports future development of machine learning tools for voice disorder classification and clinical decision support.