Evaluating the generalizability of an automated coronary artery calcium segmentation and scoring algorithm using multi-vendor dataset.

Coronary artery calcification score (CACS), also known as the Agatston score, is a significant prognostic tool for cardiovascular disease (CVD) that utilizes computed tomography (CT). We expand our previously proposed algorithm, Residual-block Inspired Coordinate Attention U-Net (RICAU-Net), to evaluate its generalizability on CT data from previously unseen scanners for lesion-specific CAC segmentation. The multi-vendor datasets were 1,108 CT scans acquired by Siemens, GE, Philips, and Toshiba. We created four groups of datasets, using data from the three scanners as the training and validation sets, while the last one as the test set to evaluate the algorithm on data from previously unseen scanners. RICAU-Net was trained using the datasets for automatic lesion-specific CAC segmentation and calcium scoring. The performance of lesion-specific segmentation and calcium scoring were evaluated using per-lesion Dice scores and intraclass correlation coefficient (ICC). And Bland-Altman plot analysis was conducted to examine the agreement between the CAC score derived from the prediction results and the ground truth. The proposed algorithm exhibited a mean absolute difference of less than 5% between the per-lesion Dice scores of the validation and test sets, indicating good generalizability on test sets comprised of data from unseen scanners during the training and validation phases. ICC analysis demonstrates that the Agatston scores calculated using predictions from RICAU-Net and manual segmentation exhibited excellent reliability at the per-patient level across all groups with ICC and 95% confidence intervals: 0.95 (0.95-0.96), 0.99 (0.99-1.00), 0.99 (0.99-0.99), and 1.00 (0.99-1.00) for group 1, 2, 3, and 4 respectively. Our algorithm demonstrates generalized performance on data from previously unseen scanners, making it potentially more suitable and practical for real-world clinical settings, where it will encounter diverse scanners from various organizations. Furthermore, a feasibility study using non-contrast chest CT scans indicates that the performance of our cardiac CT-trained algorithm on chest CT images was acceptable to a certain extent.