Deep Learning-Based Surgical Treatment Recommendation and Nonsurgical Prognosis Status Classification for Scaphoid Fractures by Automated X-ray Image Recognition.

Biomedical information retrieval for diagnosis, treatment and prognosis has been studied for a long time. In particular, image recognition using deep learning has been shown to be very effective for cancers and diseases. In these fields, scaphoid fracture recognition is a hot topic because the appearance of scaphoid fractures is not easy to detect. Although there have been a number of recent studies on this topic, no studies focused their attention on surgical treatment recommendations and nonsurgical prognosis status classification. Indeed, a successful treatment recommendation will assist the doctor in selecting an effective treatment, and the prognosis status classification will help a radiologist recognize the image more efficiently. For these purposes, in this paper, we propose potential solutions through a comprehensive empirical study assessing the effectiveness of recent deep learning techniques on surgical treatment recommendation and nonsurgical prognosis status classification. In the proposed system, the scaphoid is firstly segmented from an unknown X-ray image. Next, for surgical treatment recommendation, the fractures are further filtered and recognized. According to the recognition result, the surgical treatment recommendation is generated. Finally, even without sufficient fracture information, the doctor can still make an effective decision to opt for surgery or not. Moreover, for nonsurgical patients, the current prognosis status of avascular necrosis, non-union and union can be classified. The related experimental results made using a real dataset reveal that the surgical treatment recommendation reached 80% and 86% in accuracy and AUC (Area Under the Curve), respectively, while the nonsurgical prognosis status classification reached 91% and 96%, respectively. Further, the methods using transfer learning and data augmentation can bring out obvious improvements, which, on average, reached 21.9%, 28.9% and 5.6%, 7.8% for surgical treatment recommendations and nonsurgical prognosis image classification, respectively. Based on the experimental results, the recommended methods in this paper are DenseNet169 and ResNet50 for surgical treatment recommendation and nonsurgical prognosis status classification, respectively. We believe that this paper can provide an important reference for future research on surgical treatment recommendation and nonsurgical prognosis classification for scaphoid fractures.