Development of a multi-task deep learning system for classification of nine common knee abnormalities on MRI: a large-scale, multicentre, stepwise validation study.

Background: A comprehensive MRI-based assessment of multiple knee abnormalities is essential for guiding treatment decisions. However, knee MRI interpretation is time-consuming and prone to errors. Our study aimed to develop a deep learning system (DLS) for classifying multiple common knee abnormalities and to systematically validate its capacity in improving radiologists' diagnostic performance across multiple centres. Methods: In this multicentre study, we collected patients with knee MRI across scanners from various MRI vendors from five different centres (Centre A, The Third Affiliated Hospital of Southern Medical University; Centre B, The Sixth Affiliated Hospital of South China University of Technology; Centre C, Zhujiang Hospital of Southern Medical University; Centre D, The Fifth Affiliated Hospital of Sun Yat-sen University; and Centre E, Foshan Hospital of Traditional Chinese Medicine) between January 1, 2010 and May 31, 2022. The reference standard for knee abnormalities was established through the consensus of three musculoskeletal radiologists, based on MRI images and corresponding radiology reports. A total of 12,825 patients from Centre A were split into the training (n = 8891), validation (n = 1326), and internal test (n = 2608) sets in a 7:1:2 ratio. Patients from Centre B (n = 450) constituted the external test set I, while patients from Centres C (n = 36), D (n = 44), and E (n = 64) were combined for external test set II (n = 144). The DLS was developed with an attention-guided coarse-to-fine approach to classify nine knee abnormalities using proton density-weighted with fat suppression images in axial, coronal, and sagittal planes. This classification included primary abnormalities (meniscal tear, cartilage defect, and anterior cruciate ligament [ACL] tear) and secondary abnormalities (posterior cruciate ligament injury, medial collateral ligament [MCL] injury, lateral collateral ligament injury, infrapatellar fat pad injury, synovial plica, and cyst). A three-step validation was performed for the DLS involving 12 radiologists with varying experience levels from six geographically different hospitals (Tianhe District [Guangzhou]; Yuexiu District [Guangzhou]; Rongcheng District [Jieyang]; Nanhai District [Foshan]; Longgang District [Shenzhen]; and Wuchang District [Wuhan]) in China. Findings: This large-scale study included 13,419 patients comprising 14,962 MRI examinations for the development and validation of the DLS. The DLS demonstrated areas under the receiver operating characteristic curve (AUCs) of 0.898 (95% confidence intervals [CI], 0.890-0.905) and 0.815 (95% CI, 0.805-0.823), 0.852 (95% CI, 0.830-0.872) and 0.744 (95% CI, 0.721-0.766), and 0.812 (95% CI, 0.771-0.854) and 0.774 (95% CI, 0.733-0.816) for primary and secondary abnormalities on the internal test set, external test set I, and external test set II, respectively. For step 1 of validation, the DLS demonstrated comparable accuracy to that of the senior radiologists in classifying meniscal tear (79.6% vs. 78.1%, p = 0.426), ACL tear (89.3% vs. 89.4%, p = 1.000), and MCL injury (87.1% vs. 85.7%, p = 0.565). For step 2, the multi-reader multi-case study showed that the DLS comprehensively improved classification performance of junior (net reclassification indices [NRIs], 0.0446-0.1756, all p < 0.05) and senior radiologists (NRIs, 0.0487-0.1724, all p < 0.05) for nine knee abnormalities. For step 3, the randomised comparative study demonstrated that the junior and senior radiologists in the DLS-assisted group achieved significantly higher accuracy (difference of 4.8-8.8% and 4.2-8.2%, all p < 0.05) and reduced reading time (by 35.5 and 30.7 s, all p < 0.05) compared to the control group for nine knee abnormalities. Interpretation: This DLS demonstrated an ability to classify nine distinct knee abnormalities on MRI. When utilised alongside radiologists, classification results were accurate and performance were improved. Future prospective multi-national studies should be sought to assess this tool in real-world clinical settings. Funding: National Natural Science Foundation of China (No. 81871510, No. 82172014), the Natural Science Foundation of Guangdong Province (No. 2023A1515011318), and the Health Bureau of the Government of the Hong Kong Special Administrative Region (No. 20211021).