Deep learning for enhancing automatic classification of M-PSK and M-QAM waveform signals dedicated to single-relay cooperative MIMO 5G systems.

Automatic modulation classification (AMC) is a critical component in modern communication systems, particularly within software-defined radios, cognitive radio networks, smart grid and and distributed renewable energy systems (RESs) where adaptive and efficient signal processing is essential. This paper proposes a novel deep learning-based AMC method for identifying M-PSK and M-QAM waveform signals in single-relay cooperative MIMO 5G systems operating under partial channel state information (CSI) and spatially correlated channels. The proposed method leverages a convolutional neural network (CNN) classifier trained on a reduced set of discriminative features, including higher-order statistics and the differential nonlinear phase peak factor, which are extracted from the received signal. Feature dimensionality is reduced using the Gram-Schmidt orthogonalization procedure to enhance training efficiency. A centralized decision-making strategy aggregates predictions from multiple antennas. The method is evaluated through simulations using various modulation orders and under challenging conditions such as low signal-to-noise ratios (SNR). Results demonstrate that the proposed CNN-based approach significantly outperforms benchmark machine learning classifiers in terms of classification accuracy, precision, recall, and F-measure. These findings underscore the practical potential of the method for enhancing AMC performance in realistic 5G cooperative scenarios.