A personalized automated system of 3D facial soft tissue landmarks annotation based on deep learning and computer vision.

objectiveThis study aims to develop a personalized automated system for annotating three-dimensional (3D) facial soft tissue landmarks utilizing deep learning and computer vision. By comparing the results of automated annotations with manual annotations, the accuracy and clinical applicability of the proposed algorithm were systematically evaluated, providing an efficient and precise analysis system for facial morphology research.

methodsA total of 55 Chinese orthodontic patients (24 males, 31 females; mean age 23.4 ± 7.01 years) were recruited from Shandong University Stomatological Hospital, comprising 40 patients with normal facial morphology and 15 with severe craniofacial deformities (mandibular asymmetry, severe skeletal Class II and III; n = 5 each). This study constructed a personalized automated system based on deep learning and computer vision. Through standardized facial template construction with 68 key points, automated 68-landmark annotation of original scans, 3D facial nonlinear registration, and personalized keypoint transfer, the system enables one-time personalized annotation of the standard template and subsequent automatic batch mapping to multiple facial scan models. Deformity-specific personalized templates were constructed for severe malocclusions. Annotation accuracy was evaluated against manual annotations by experienced orthodontic experts using 21 clinically relevant landmarks.

resultsThe system demonstrated outstanding accuracy with an average Euclidean distance of 0.95 ± 0.48 mm for 21 selected landmarks, surpassing most existing algorithms. Proportion-based analysis revealed that 96.3% of all landmarks achieved clinically acceptable accuracy within 2.0 mm, with 84.0% within 1.5 mm and 57.0% achieving sub-millimeter precision. The average errors for linear and angular measurements were 0.31 ± 0.35 mm and 1.05 ± 0.90°, respectively. For mandibular deformity samples, personalized templates significantly improved accuracy, reducing mean Euclidean distance from 0.95 ± 0.33 mm (standardized template) to 0.84 ± 0.21 mm (personalized template), with significant error reductions across all 21 landmarks, particularly in mandibular regions.

conclusionsThe proposed automated annotation system achieves efficient and precise annotation of 3D facial soft tissue landmarks through standardized facial template construction, 3D facial nonlinear registration, and personalized landmark transfer. This system provided significant convenience for the measurement of three-dimensional landmarks of soft tissues, the analysis of facial soft tissue morphology, and the diagnosis, treatment, and efficacy evaluation of malocclusion. Additionally, The personalized template approach enhances annotation precision for severe facial deformities, broadening clinical applicability.