A comparative evaluation of gradient-based optimization algorithms for short-term load forecasting using deep residual networks.

Short-Term Load Forecasting (STLF) is essential for the reliable and economic operation of modern power systems. Deep Residual Networks (DRNs) have emerged as an effective framework for STLF due to their ability to model nonlinear and multi-scale load patterns. Although numerous DRN-based extensions have been proposed through architectural refinement and feature enhancement, the role of gradient-based optimization algorithms in DRN-based STLF has received limited systematic investigation. Most existing studies rely on the Adaptive Moment Estimation (Adam) algorithm as the default optimization strategy, without comprehensively examining alternative gradient-based optimizers. To address this gap, this study conducts a hypothesis-driven comparative evaluation of representative gradient-based optimization algorithms within a unified DRN-based STLF framework across both temperate (ISO-NE) and tropical (MyPJ) climatic conditions. Both the original DRN, which primarily incorporates temperature as the meteorological input, and its enhanced variant, the Principal Component Analysis–Deep Residual Network (PCA-DRN), which integrates multiple weather variables through PCA, are investigated using real-world electricity load datasets. Forecasting performance is evaluated using multiple metrics, with Mean Absolute Percentage Error (MAPE) as the primary criterion, and statistical significance is assessed through a nonparametric bootstrap resampling procedure. The results demonstrate that optimizer selection significantly influences training stability and forecasting accuracy. AMSGrad achieves the most consistent performance within the original DRN across climatic conditions, whereas under PCA-based feature representation the relative advantage shifts, indicating that meteorological feature compression reshapes the optimization landscape. Overall, the findings highlight the importance of systematic optimizer evaluation and feature-representation strategies for enhancing the reliability and stability of DRN-based STLF.