In the era of big data, reducing data dimensionality is critical in many areas of science. Widely used Principal Component Analysis (PCA) addresses this problem by computing a low dimensional data embedding that maximally explain variance of the data. However, PCA has two major weaknesses. Firstly, it only considers linear correlations among variables (features), and secondly it is not suitable for categorical data. We resolve these issues by proposing Maximally Correlated Principal Component Analysis (MCPCA). MCPCA computes transformations of variables whose covariance matrix has the largest Ky Fan norm. Variable transformations are unknown, can be nonlinear and are computed in an optimization. MCPCA can also be viewed as a multivariate extension of Maximal Correlation. For jointly Gaussian variables we show that the covariance matrix corresponding to the identity (or the negative of the identity) transformations majorizes covariance matrices of non-identity functions. Using this result we characterize global MCPCA optimizers for nonlinear functions of jointly Gaussian variables for every rank constraint. For categorical variables we characterize global MCPCA optimizers for the rank one constraint based on the leading eigenvector of a matrix computed using pairwise joint distributions. For a general rank constraint we propose a block coordinate descend algorithm and show its convergence to stationary points of the MCPCA optimization. We compare MCPCA with PCA and other state-of-the-art dimensionality reduction methods including Isomap, LLE, multilayer autoencoders (neural networks), kernel PCA, probabilistic PCA and diffusion maps on several synthetic and real datasets. We show that MCPCA consistently provides improved performance compared to other methods.
翻译:在大数据时代, 降低数据维度是许多科学领域的关键。 广泛使用的主元元元分析( PCA) 计算低维数据嵌入, 以最充分地解释数据差异来解决这个问题。 但是, CPA有两大弱点。 首先, 它只考虑变量( 特性) 之间的线性关联, 其次它不适合绝对数据 。 我们通过提出与最大高度相关的主元元元分析( MCPCA) 来解决这些问题 。 MCPCA 计算变量的变化, 这些变量的变量的变异具有最大的 Ky Fan 标准。 变量的变异性是未知的, 可以是非线性化的, 也可以在优化中进行计算。 MICCA 也可以被视为最大正统的多维化扩展。 对于共同的变异性变量, 我们共同的MACA 最大正统的变异性矩阵, 显示一个比值的压力 。