Enhanced breast cancer detection in histopathology images using graph transformer GANs with graph masked modeling and osprey optimization.

Breast cancer (BC) diagnosis from histopathology images requires accurate identification of malignant tissue while accounting for complex tissue structures and variations in image magnification. In order to overcome these problems, this study suggests GTGAN-GMM-OOA, a graph-based framework that jointly models spatial relationships among tissue regions, learns masked graph representations, and adaptively optimizes model parameters. Unlike conventional image-based approaches that primarily focus on independent visual features, the proposed framework explicitly captures structural dependencies between tissue regions through graph transformer learning. It strengthens representation learning through graph-masked modeling. The Osprey Optimization Algorithm (OOA) is further employed to optimize the model configuration. ISGIF pre-processing and SMT-U-Net++ are introduced to improve image quality and learn discriminative tissue features. Using the BreakHis database, the proposed technique is capable of obtaining 99.9%, 99.7%, 99.9%, and 99.5% of accuracies for 40X, 100X, 200X, and 400X magnification levels, respectively. Additionally, five-fold cross validation is used to validate the stability of the developed technique, with an average accuracy of 99.75% and standard deviation of ±0.12%. The obtained results prove the efficiency and effectiveness of the developed technique for graph-based BC histopathology classification.