Fast and Scalable Estimator for Sparse and Unit-Rank Higher-Order Regression Models

Because tensor data appear more and more frequently in various scientific researches and real-world applications, analyzing the relationship between tensor features and the univariate outcome becomes an elementary task in many fields. To solve this task, we propose \underline{Fa}st \underline{S}parse \underline{T}ensor \underline{R}egression model (FasTR) based on so-called unit-rank CANDECOMP/PARAFAC decomposition. FasTR first decomposes the tensor coefficient into component vectors and then estimates each vector with $\ell_1$ regularized regression. Because of the independence of component vectors, FasTR is able to solve in a parallel way and the time complexity is proved to be superior to previous models. We evaluate the performance of FasTR on several simulated datasets and a real-world fMRI dataset. Experiment results show that, compared with four baseline models, in every case, FasTR can compute a better solution within less time.

[1]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[2]  Wenwen Li,et al.  Sturm: Sparse Tubal-Regularized Multilinear Regression for fMRI , 2018, MLMI@MICCAI.

[3]  Rose Yu,et al.  Tensor Regression Meets Gaussian Processes , 2017, AISTATS.

[4]  Han Liu,et al.  Provable sparse tensor decomposition , 2015, 1502.01425.

[5]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[6]  Dinggang Shen,et al.  Multiscale adaptive regression models for neuroimaging data , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[7]  Kohei Hayashi,et al.  Doubly Decomposing Nonparametric Tensor Regression , 2015, ICML.

[8]  Jimeng Sun,et al.  Model-Driven Sparse CP Decomposition for Higher-Order Tensors , 2017, 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[9]  Hongtu Zhu,et al.  Tensor Regression with Applications in Neuroimaging Data Analysis , 2012, Journal of the American Statistical Association.

[10]  Haiping Lu,et al.  Multilinear Regression for Embedded Feature Selection with Application to fMRI Analysis , 2017, AAAI.

[11]  Lexin Li,et al.  STORE: Sparse Tensor Response Regression and Neuroimaging Analysis , 2016, J. Mach. Learn. Res..

[12]  Genevera I. Allen,et al.  Sparse Higher-Order Principal Components Analysis , 2012, AISTATS.

[13]  Garvesh Raskutti,et al.  Convex regularization for high-dimensional multiresponse tensor regression , 2015, The Annals of Statistics.

[14]  W. Art Chaovalitwongse,et al.  Sparse optimization in feature selection: application in neuroimaging , 2014, J. Glob. Optim..

[15]  Jian Yang,et al.  Sparse Tensor Additive Regression , 2019, J. Mach. Learn. Res..

[16]  Pradeep Ravikumar,et al.  Elementary Estimators for High-Dimensional Linear Regression , 2014, ICML.

[17]  Jiayu Zhou,et al.  Boosted Sparse and Low-Rank Tensor Regression , 2018, NeurIPS.

[18]  Weiwei Guo,et al.  Tensor Learning for Regression , 2012, IEEE Transactions on Image Processing.

[19]  Misha Elena Kilmer,et al.  Novel Methods for Multilinear Data Completion and De-noising Based on Tensor-SVD , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  V. Strassen Gaussian elimination is not optimal , 1969 .

[21]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[22]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[23]  James G. Scott,et al.  Tensor Decomposition With Generalized Lasso Penalties , 2015, 1502.06930.

[24]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[25]  F. L. Hitchcock The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[26]  Zemin Zhang,et al.  Exact Tensor Completion Using t-SVD , 2015, IEEE Transactions on Signal Processing.

[27]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[28]  Rose Yu,et al.  Learning from Multiway Data: Simple and Efficient Tensor Regression , 2016, ICML.

[29]  Hongtu Zhu,et al.  Intrinsic Regression Models for Positive-Definite Matrices With Applications to Diffusion Tensor Imaging , 2009, Journal of the American Statistical Association.

[30]  Xu Tan,et al.  Logistic Tensor Regression for Classification , 2012, IScIDE.

[31]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .