A hybrid generative and transformer-based framework for anomaly detection in industrial sensor time-series for predictive maintenance.

The multivariate time-series sensor data present in Industry 4.0 generates a number of difficulties for anomaly detection for predictive maintenance, including small fault datasets with imbalanced distributions and dynamically evolving industrial environments. The hybrid anomaly detection framework suggested in this paper aims to provide a solution for both unsupervised and supervised versions of industrial anomaly detection by combining improved TransGAN architectures, transformer-based temporal sequence modelling and statistical validation. The framework includes a range of techniques including convolutional feature extraction, multi-headed self-attention transformer blocks, Bidirectional LSTM-GRU temporal learning, adversarial reconstruction learning, and weighted fusion of reconstruction-based and discriminator-based scores to detect anomalies. The proposed framework has been evaluated on two benchmark datasets, including the 'Machine Failure' dataset and the 'SWaT' (Secure Water Treatment) dataset and on a real-world unlabeled crane current data set, which provided a generalised evaluation across different industrial domains. On the Machine Failure dataset, the experimental results obtained were: 84.09% accuracy; 68.95% precision; 84.32% recall; and 75.86% F1-score. For the SWaT dataset, the corresponding values were: 92.25% accuracy 66.65% precision 72.33% recall and 69.37% F1-score. For the unlabeled crane dataset, various methods including GAN, Anomaly Transformer, MSPG-SEN, and statistical validation (i.e. STUMPY, Merlion) are used to conduct unsupervised anomaly detection. Instead of using predictions from each method independently, the authors proposed a weighted fusion and cross-model agreement approach to increase reliability by overlapping detections and leveraging different types of anomalies. The hybrid fusion framework consistently produced 14 high-confidence anomalies that were supported by a variety of deep learning and statistical paradigms. The combination of generative modeling, transformer-based temporal dependency modeling, and statistical validation methods proved to be a more reliable way to detect anomalies than any individual method alone. The proposed approach will provide an effective means to perform predictive maintenance in an industrial environment where there is limited labeled data and the process of validating faults is difficult.