Clinical scale of artificial intelligence-based nerve segmentation on ultrasound: a pilot validation study.

Background: The objective metrics for AI-based nerve segmentation are time-consuming and may not fully reflect their clinical usefulness. We aimed to 1) conduct a pilot validation study of the Clinical Segmentation Evaluation Scale (C-SES), as an estimation for an objective metric, and 2) establish a threshold on those scales, because the thresholds were derived for both the new C-SES scale AND for objective metrics (IoU, DSC) to determine the usefulness of AI predictions. Methods: Seven experts rated 74 ultrasound sequences twice using AI-based nerve recognition software (cNerve) in 3 anatomical regions. In Step 1, the C-SES was evaluated for 1) content validity (expert appraisal of relevance, comprehensiveness, and clarity), 2) concurrent criterion validity (by correlating the C-SES with a reference standard, the Intersection over Union-IoU), and 3) reliability (intra and inter-rater) and error measurement. In an additional Step 2, expert-perceived usefulness for non-experts and novice anesthesiologists was evaluated to derive candidate thresholds for Intersection over Union (IoU) and C-SES metrics within the investigated clinical context. Results: In Step 1, content validity was preliminarily supported by expert consensus. Concurrent criterion validity was strong (Pearson's Conclusion: In this pilot validation, the C-SES appeared to estimate objective segmentation metrics, although further validation is needed. The proposed thresholds are preliminary and hypothesis-generating rather than universal clinical benchmarks and require confirmation before generalization beyond expert assessors and this spectrum-enriched sample.