Forward Stepwise Deep Autoencoder-based Monotone Nonlinear Dimensionality Reduction Methods.

Dimensionality reduction is an unsupervised learning task aimed at creating a low-dimensional summary and/or extracting the most salient features of a dataset. Principal components analysis (PCA) is a linear dimensionality reduction method in the sense that each principal component is a linear combination of the input variables. To allow features that are nonlinear functions of the input variables, many nonlinear dimensionality reduction methods have been proposed. In this paper we propose novel nonlinear dimensionality reduction methods based on bottleneck deep autoencoders (Kramer, 1991). Our contributions are two-fold: (1) We introduce a monotonicity constraint into bottleneck deep autoencoders for estimating a single nonlinear component and propose two methods for fitting the model. (2) We propose a new, forward stepwise (FS) deep learning architecture for estimating multiple nonlinear components. The former helps extract interpretable, monotone components when the assumption of monotonicity holds, and the latter helps evaluate reconstruction errors in the original data space for a range of components. We conduct numerical studies to compare different model fitting methods and use two real data examples from the studies of human immune responses to HIV to illustrate the proposed methods.