Semi-automated end-to-end quality control system for annotation data based on automated Cobb angle measurement.

purposeThe field of automated scoliosis diagnosis necessitates large-scale data annotation; however, existing annotation practices are hampered by the lack of a systematic, end-to-end quality control framework. To address this gap, we aim to develop a Semi-Automated End-to-End QC (SAE-QC) System designed to ensure annotation accuracy while substantially accelerating the data collection pipeline and reducing operational costs. This system is expected to provide critical support for the construction of high-quality datasets and the development of reliable AI models in the domain of automated Cobb angle measurement.

methodsWe propose the SAE-QC System for medical image annotation, featuring core modules including a data assignment module with dual-blind annotation pairing medical and non-medical annotators, an annotation protocol QC module enforcing standardized guidelines through structured validation, and an annotation QC module performing comprehensive quality checks from format validation to multi-annotator consistency analysis. The system’s modular architecture ensures cross-platform compatibility and seamless integration across the entire annotation workflow from data distribution to expert adjudication, while maintaining flexibility for different medical imaging tasks and data formats.

resultsWe developed the SAE-QC System, comprising a Data Assignment Module, an Annotation Protocol QC Module, and an Annotation QC Module. By integrating automated algorithms with expert supervision, the system enables end-to-end management of both Cobb angle measurement and vertebral segmentation, significantly reducing error rates and improving data reliability. When applied to a dataset construction task, experimental results showed the following improvements in annotation quality: the abnormality rate for Cobb angle measurement decreased from 66% to 19%, measurement consistency improved by approximately 40%, and the abnormality rate for vertebral segmentation decreased from 27% to 2.9%, with a significant increase in overall accuracy.

conclusionThis study demonstrates that the SAE-QC system—integrating automated quality assurance with expert supervision—effectively enhances the reliability of medical image annotation, providing a practical framework for non-specialist teams to achieve high-quality annotations.