Radiographic classification of mandibular osteoradionecrosis: A blinded prospective multi-disciplinary interobserver diagnostic performance study.

Background: Osteoradionecrosis of the jaw (ORNJ) is a debilitating complication that affects up to 15% of head and neck cancer patients who undergo radiotherapy. The ASCO/ISOO/MASCC-endorsed ClinRad severity classification system was recently proposed (and recommended in the latest ASCO guidelines) to incorporate radiographic findings for determining ORNJ severity based on the vertical extent of bone necrosis. However, variability in imaging modalities and specialty-specific knowledge may contribute to disparities in diagnosing and classifying ORNJ. This study aims to evaluate and benchmark multi-specialty physician performance in diagnosing and severity classification of ORNJ using different radiographic imaging. Methods: A single institution retrospective diagnostic validation study was conducted at The University of Texas MD Anderson Cancer Center involving 20 healthcare providers across varying specialties including oral oncology, radiation oncology, surgery, and neuroradiology. Participants reviewed 85 de-identified imaging sets including computed tomography (CT) and orthopantomogram (OPG) images from 30 patients with confirmed ORN, with blinded replicates (n=10) for assessment of intra-observer variability and asked to diagnose and stage ORNJ using the ClinRad system. Diagnostic performance was assessed using ROC curves; intra- and inter-observer agreement were measured with Cohen's and Fleiss kappa, respectively. Sub-analyses considered physician specialty, years of clinical experience and level of confidence. Results: Paired CT-OPG imaging improved ORNJ diagnostic performance across all specialties, with AUC values ranging from 0.79 (residents) to 0.98 (surgeons). Inter- and intra-rater agreements for ORNJ detection were limited, with median (IQR) Fleiss and Cohen's kappa values of 0.38 (0.22) and 0.08 (0.17), respectively. Slight to fair inter-rater agreement in severity classification ORNJ was observed with median (IQR) Fleiss kappa values of 0.22, 0.13, and 0.05 for stages 0/1, 2, and 3, respectively. The most commonly reported radiographic features for confirmed ORNJ cases staged as ClinRad grade 1 or 2 were "bone necrosis confined to alveolar bone" (22.7%), "bone necrosis involving the basilar bone or maxillary sinus" (14.8%), and "bone lysis/sclerosis" (20.0%). Conclusion: This study establishes an essential benchmark for physician detection of radiographic ORNJ. The significant variability in diagnostic and severity classification observed across specialties emphasizes the need for standardized imaging protocols and specialist training as well as highlights the value of multimodality imaging.