Automated Multi-class Segmentation of Mandibular Odontogenic Cysts and the Mandibular Canal on CBCT Images Using a Hybrid Deep Learning Framework.

Mandibular odontogenic cysts can complicate surgical planning if located near the mandibular canal. Accurate delineation of both structures is therefore clinically important. This study presents a deep learning-based method for simultaneously segmenting mandibular odontogenic cysts and the mandibular canal in cone-beam computed tomography (CBCT) images. The developed method relies on a hybrid deep learning architecture integrating a ConvNeXt-Base encoder, a Dilated Context Block, a U-Net-based decoder, and a Deep Supervision mechanism. This framework simultaneously segments the mandibular canal and odontogenic cyst regions within the same architecture, promoting multi-scale contextual feature learning. We evaluated the performance of the proposed method using fivefold cross-validation experiments on a real-world clinical CBCT dataset comprising 103 patient images. We assessed segmentation performance using overlap-based metrics together with boundary-aware HD95 and ASSD metrics. The experimental results demonstrated that the model achieved a mean intersection over union (mIoU) of 76.28% ± 1.83%. Excluding the background class yielded macro IoU and macro F1-score values of 64.46% ± 2.74% and 77.00% ± 1.89%, respectively. Meanwhile, the weighted IoU and weighted F1-score values were 78.71% ± 4.25% and 87.63% ± 2.63%, respectively. In boundary-based evaluations, the mandibular canal had an HD95 of 2.44 ± 2.98 mm and an ASSD of 0.45 ± 0.30 mm. The corresponding values for odontogenic cysts were 5.59 ± 3.65 mm and 1.43 ± 0.88 mm, respectively. These results suggest that the mandibular canal is more consistently delineated, while cyst segmentation remains more challenging, particularly at irregular and low-contrast boundaries. The proposed framework allows for the simultaneous and objective segmentation of mandibular odontogenic cysts and the mandibular canal within a unified, multi-class CBCT analysis framework. By providing reproducible delineation of both structures, this method could serve as a technical foundation for future quantitative analyses of lesion-canal relationships. However, further validation using larger, multicenter, volumetric CBCT datasets is required before clinical translation.