Formal validation of a deep learning-based automated interpretation system for cardiac structure and function in adult echocardiography.

Background: Accurate measurement of cardiac structure and function is the basis of diagnosis of cardiac diseases, but it is time-consuming and empirically-dependent. This study attempted to propose a deep learning (DL) interpretation of cardiac structure and function. Methods: The training dataset consisted of 416 video loops and 892 Doppler images drawn from 141 patients undergoing clinical echocardiography from 2020 to 2021. Two experts labeled these images using the Pair platform. From this, DL algorithms including the Auto-Echo and Auto-Doppler were trained to measure echocardiographic parameters. Subsequently, eight sonographers with different years of echocardiographic experience labeled a validation dataset of 178 new video loops and 391 Doppler images obtained from 60 new patients. One highly trained expert annotated the external validation dataset of 90 two-dimensional (2D) videos and 120 Doppler images. The standard deviation ratio (SD ratio), Bland-Altman analysis, interclass correlation coefficient (ICC), mean absolute deviation (MAD), absolute relative deviation, and correlation analysis were employed to investigate the agreement between DL and human experts. Results: For the structure parameters' measurements including four-chamber dimensions, the SD ratios ranged from 0.70 to 1.02, and the ICCs showed that automated measurements were equivalent or superior to human expert measurements. The correlation coefficients were greater than 0.85 for 83.3% of the parameters, the MADs ranged from -1.5 to 1.9 mm, and the absolute relative deviations ranged from 2.5% to 9.7%. However, large absolute deviations were observed for parameters in RV-A4C view and RV, which was consistent with human readers. For Doppler parameters, including four transvalvular velocity measurements, the correlation coefficients ranged from 0.81 to 0.99, the absolute relative deviation of all pulse Doppler parameters was within 10%, and 100% (9/9) of tissue Doppler parameters were within 5%. However, the velocity-time integral (VTI) of transvalvular velocity showed large absolute relative deviations between the automated and manual measurements. Auto-Echo saved 95.4% and Auto-Doppler saved 82.5% analysis time upon human experts. In the external validation cohort, the mean absolute relative deviation for almost all structural parameters and Doppler parameters was within 10%. Conclusions: The measurements of our DL interpretation had high accuracy, increased efficiency of the examination, and were inter-changeable with human experts' assessment. It has shown human-like patterns of measurements, as the same trend of difference can be observed between DL and different experienced readers.