Fast Circulant Tensor Power Method for High-Order Principal Component Analysis

To understand high-order intrinsic key patterns in high-dimensional data, tensor decomposition is a more versatile tool for data analysis than standard flat-view matrix models. Several existing tensor models aim to achieve rapid computation of high-order principal components based on the tensor power method. However, since a tensor power method does not enforce orthogonality in subsequently calculated decomposition components, it causes far more challenges on principal component analysis of high-order tensors. To address this problem, several tensor power method variant algorithms incorporating sparsity into decomposition factors have been proposed. However, because these variant algorithms require additional procedures based on data-driven hyper-parameter optimization algorithms, a trade-off between computational cost and convergence exists. In this paper, a novel tensor power method called the fast circulant tensor power method is proposed. The proposed algorithm combines tensor-train decomposition and the power method. Tensor-train decomposition is a high-order tensor decomposition method based on auxiliary unfolding matrix decomposition. Thus, the power method can be embedded into our methodology without any additional processes. Notably, a simple combination of these two methods may cause a local optima problem because the power method only guarantees convergence on each unfolding matrix in tensor-train decomposition. To solve this problem, the circulant updating method is proposed, which globally optimizes all factor vectors by reordering some steps of the factor vector updates. It is experimentally demonstrated that, compared to state-of-the-art tensor power method variant methodologies, the proposed algorithm achieves the lowest computational complexity and quantitatively good performance in various applications including large-scale color image decomposition and convolutional neural network compression.