Auto-segmentation of organs-of-interest clinical acceptability & reproducibility framework in head and neck cancer.

Background and purpose: Auto-segmentation of organs-of-interest (OOI) in cancer patients is essential for facilitating radiotherapy planning and reducing inter-observer variability. Deep learning-based auto-segmentation models have shown promise, but limited transparency and reproducibility hinder their generalizability and clinical acceptability, limiting their use in clinical settings. Materials and methods: We introduced an auto-Segmentation Clinical Acceptability & Reproducibility Framework, a comprehensive framework designed to benchmark open-source deep learning models for auto-segmentation of 19 essential OOIs in head and neck cancer (HNC). Reproducibility was achieved through harmonized data curation, standardized model training (training/tune/test of 479/44/59) and assessment workflows. New models can be benchmarked against 12 pre-trained open-source deep learning models, while estimating clinical acceptability using a 5-point Likert scale. Results: The framework codebase is openly available for benchmarking OOI auto-segmentation methods. During development, expert assessment of the best performing model labelled 16/19 AI-generated OOI categories as clinically acceptable with only minor revisions. Conclusions: The framework facilitates benchmarking and expert assessment of AI-driven auto-segmentation tools, addressing the need for transparency and reproducibility in this domain. Through its emphasis on clinical acceptability, our framework fosters the integration of AI models into clinical environments, specifically within radiation therapy.