Encoder-Decoder Full Residual Deep Networks for Robust Regression and Spatiotemporal Estimation.

Although increasing hidden layers can improve the ability of a neural network in modeling complex nonlinear relationships, deep layers may result in degradation of accuracy due to the problem of vanishing gradient. Accuracy degradation limits the applications of deep neural networks to predict continuous variables with a small sample size and/or weak or little invariance to translations. Inspired by residual convolutional neural network in computer vision, we developed an encoder-decoder full residual deep network to robustly regress and predict complex spatiotemporal variables. We embedded full shortcuts from each encoding layer to its corresponding decoding layer in a systematic encoder-decoder architecture for efficient residual mapping and error signal propagation. We demonstrated, theoretically and experimentally, that the proposed network structure with full residual connections can successfully boost the backpropagation of signals and improve learning outcomes. This novel method has been extensively evaluated and compared with four commonly used methods (i.e., plain neural network, cascaded residual autoencoder, generalized additive model, and XGBoost) across different testing cases for continuous variable predictions. For model evaluation, we focused on spatiotemporal imputation of satellite aerosol optical depth with massive nonrandomness missingness and spatiotemporal estimation of atmospheric fine particulate matter [Formula: see text] (PM