Fine-grained classification of thoracic vertebral compression fractures based on multi-layer feature fusion and attention-guided patch recombination.

The distinction between benign and malignant thoracic vertebral compression fractures (VCFs) on magnetic resonance imaging (MRI) is often subtle, and distinguishing between them is a fine-grained classification challenge. We propose a method for benign and malignant classification of thoracic VCFs based on multi-layer feature fusion and attention-guided patch reorganization to address the problem of low-level feature loss and noise associated with computing background patches when vision transformer (ViT) is applied to the task of fine-grained MRI image classification. The approach is based on the ViT architecture, which fuses low-level features with high-level features by selecting discriminative tokens using multiple layers of mutual attention weights between the classification tokens and the tokens. In addition, we incorporate an attention-guided patch recombination module that uses attention weights to select and combine patches of any two input images, which enhances the richness of the input images while reducing the noise computation. Experiments were conducted on the thoracic VCFs dataset with quantitative assessment metrics, achieving slice-level classification accuracy and AUC of 84.87% and 84.19%, respectively. Aggregating slice-level predictions, the patient-level classification accuracy reached 93.18%. Compared to other fine-grained ViT-based methods, our approach demonstrated varying improvements in slice-level accuracy, with a maximum increase of 3.65%. Ablation experiments further validated the effectiveness of the multi-layer feature fusion and patch recombination modules. The proposed MFAR-ViT utilizes the advantages of multi-layer feature fusion and patch reorganization to identify the fine-grained differences in MRI images of thoracic VCFs more efficiently, which is expected to help doctors diagnose the patient's condition quickly and accurately.