L1-Norm Tucker Tensor Decomposition

Tucker decomposition is a standard multi-way generalization of Principal-Component Analysis (PCA), appropriate for processing tensor data. Similar to PCA, Tucker decomposition has been shown to be sensitive against faulty data, due to its L2-norm-based formulation which places squared emphasis to peripheral/outlying entries. In this work, we explore L1-Tucker, an L1-norm based reformulation of Tucker decomposition, and present two algorithms for its solution, namely L1-norm Higher-Order Singular Value Decomposition (L1-HOSVD) and L1-norm Higher-Order Orthogonal Iterations (L1-HOOI). The proposed algorithms are accompanied by complexity and convergence analysis. Our numerical studies on tensor reconstruction and classification corroborate that L1-Tucker decomposition, implemented by means of the proposed algorithms, attains similar performance to standard Tucker when the processed data are corruption-free, while it exhibits sturdy resistance against heavily corrupted entries.

[1]  Andrzej Cichocki,et al.  One time is not enough: iterative tensor decomposition for neural network compression , 2019, ArXiv.

[2]  Rafael T. de Sousa,et al.  Tensor-Based Channel Estimation for Massive MIMO-OFDM Systems , 2019, IEEE Access.

[3]  Panos P. Markopoulos,et al.  Adaptive L1-Norm Principal-Component Analysis With Online Outlier Rejection , 2018, IEEE Journal of Selected Topics in Signal Processing.

[4]  Martin Haardt,et al.  HOSVD-Based Denoising for Improved Channel Prediction of Weak Massive MIMO Channels , 2017, 2017 IEEE 85th Vehicular Technology Conference (VTC Spring).

[5]  Panos P. Markopoulos,et al.  L1-fusion: Robust linear-time image recovery from few severely corrupted copies , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[6]  Panos P. Markopoulos,et al.  INCREMENTAL COMPLEX L1-PCA FOR DIRECTION-OF-ARRIVAL ESTIMATION , 2018, 2018 IEEE Western New York Image and Signal Processing Workshop (WNYISPW).

[7]  Panos P. Markopoulos,et al.  Indoor human motion classification by L1-norm subspaces of micro-Doppler signatures , 2017, 2017 IEEE Radar Conference (RadarConf).

[8]  Panos P. Markopoulos,et al.  L1-NORM HIGHER-ORDER SINGULAR-VALUE DECOMPOSITION , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[9]  Andrzej Cichocki,et al.  Efficient Nonnegative Tucker Decompositions: Algorithms and Uniqueness , 2014, IEEE Transactions on Image Processing.

[10]  Gene H. Golub,et al.  Matrix computations , 1983 .

[11]  Matthieu Cord,et al.  MUTAN: Multimodal Tucker Fusion for Visual Question Answering , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[13]  Panos P. Markopoulos,et al.  Efficient L1-Norm Principal-Component Analysis via Bit Flipping , 2016, IEEE Transactions on Signal Processing.

[14]  Martin Haardt,et al.  Tensor-based approach to channel estimation in amplify-and-forward MIMO relaying systems , 2014, 2014 IEEE 8th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[15]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[16]  Gérard Favier,et al.  Tensor-Based Joint Channel and Symbol Estimation for Two-Way MIMO Relaying Systems , 2019, IEEE Signal Processing Letters.

[17]  Nikos D. Sidiropoulos,et al.  Tensors for Data Mining and Data Fusion , 2016, ACM Trans. Intell. Syst. Technol..

[18]  Arie Yeredor,et al.  First-order perturbation analysis of low-rank tensor approximations based on the truncated HOSVD , 2016, 2016 50th Asilomar Conference on Signals, Systems and Computers.

[19]  Panos P. Markopoulos,et al.  The Exact Solution to Rank-1 L1-Norm TUCKER2 Decomposition , 2017, IEEE Signal Processing Letters.

[20]  Panos P. Markopoulos,et al.  Options for multimodal classification based on L1-Tucker decomposition , 2019, Big Data: Learning, Analytics, and Applications.

[21]  Feiping Nie,et al.  Robust Principal Component Analysis with Non-Greedy l1-Norm Maximization , 2011, IJCAI.

[22]  Maja Pantic,et al.  TensorLy: Tensor Learning in Python , 2016, J. Mach. Learn. Res..

[23]  Chris H. Q. Ding,et al.  Robust Tucker Tensor Decomposition for Effective Image Representation , 2013, 2013 IEEE International Conference on Computer Vision.

[24]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[25]  Florian Roemer,et al.  Higher-Order SVD-Based Subspace Estimation to Improve the Parameter Estimation Accuracy in Multidimensional Harmonic Retrieval Problems , 2008, IEEE Transactions on Signal Processing.

[26]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[27]  Ying Liu,et al.  Compressed-Sensed-Domain L1-PCA Video Surveillance , 2015, IEEE Transactions on Multimedia.

[28]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[29]  Breton Minnehan,et al.  Grassmann Manifold Optimization for Fast $L_1$-Norm Principal Component Analysis , 2019, IEEE Signal Processing Letters.

[30]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[31]  Dimitris A. Pados,et al.  Video background tracking and foreground extraction via L1-subspace updates , 2016, SPIE Commercial + Scientific Sensing and Imaging.

[32]  Christos Faloutsos,et al.  S-HOT: Scalable High-Order Tucker Decomposition , 2017, WSDM.

[33]  Han Liu,et al.  Challenges of Big Data Analysis. , 2013, National science review.

[34]  H. Nkansah.  Least squares optimization with L1-norm regularization , 2017 .

[35]  Jieping Ye,et al.  Generalized Low Rank Approximations of Matrices , 2005, Machine Learning.

[36]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[38]  Yousef Saad,et al.  Higher Order Orthogonal Iteration of Tensors (HOOI) and its Relation to PCA and GLRAM , 2007, SDM.

[39]  Hong Cheng,et al.  Generalized Higher-Order Orthogonal Iteration for Tensor Decomposition and Completion , 2014, NIPS.

[40]  Lei Wang,et al.  Generalized 2D principal component analysis for face image representation and recognition , 2005, Neural Networks.

[41]  Donald Goldfarb,et al.  Robust Low-Rank Tensor Recovery: Models and Algorithms , 2013, SIAM J. Matrix Anal. Appl..

[42]  Panos P. Markopoulos,et al.  Robust decomposition of 3-way tensors based on L1-norm , 2018, Commercial + Scientific Sensing and Imaging.

[43]  Stephen Becker,et al.  Low-Rank Tucker Decomposition of Large Tensors Using TensorSketch , 2018, NeurIPS.

[44]  Pierre Comon,et al.  Some Theory on Non-negative Tucker Decomposition , 2017, LVA/ICA.

[45]  Nikos D. Sidiropoulos,et al.  Joint Tensor Factorization and Outlying Slab Suppression With Applications , 2015, IEEE Transactions on Signal Processing.

[46]  Tamara G. Kolda,et al.  Parallel Tensor Compression for Large-Scale Scientific Data , 2015, 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[47]  Xuelong Li,et al.  Robust Tensor Analysis With L1-Norm , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[48]  Panos P. Markopoulos,et al.  L1-Norm Principal-Component Analysis of Complex Data , 2017, IEEE Transactions on Signal Processing.

[49]  Wei Liu,et al.  Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Panos P. Markopoulos,et al.  Optimal Algorithms for L1-subspace Signal Processing , 2014, IEEE Transactions on Signal Processing.

[51]  Raf Vandebril,et al.  A New Truncation Strategy for the Higher-Order Singular Value Decomposition , 2012, SIAM J. Sci. Comput..

[52]  Yangyang Xu,et al.  On the convergence of higher-order orthogonal iteration , 2015 .

[53]  Jieping Ye,et al.  Generalized Low Rank Approximations of Matrices , 2004, Machine Learning.

[54]  W. Marsden.  I and J , 2012 .

[55]  Yimin Wei,et al.  Randomized algorithms for the approximations of Tucker and the tensor train decompositions , 2018, Advances in Computational Mathematics.

[56]  J. Tropp,et al.  Two proposals for robust PCA using semidefinite programming , 2010, 1012.1086.

[57]  Hairong Qi,et al.  Low-rank tensor decomposition based anomaly detection for hyperspectral imagery , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[58]  Danilo P. Mandic,et al.  Tucker Tensor Layer in Fully Connected Neural Networks , 2019, ArXiv.

[59]  Yang Guo,et al.  Low-Rank Tucker Approximation of a Tensor From Streaming Data , 2019, SIAM J. Math. Data Sci..

[60]  Piya Pal,et al.  On canonical polyadic decomposition of overcomplete tensors of arbitrary even order , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[61]  A. Cichocki,et al.  Tensor decompositions for feature extraction and classification of high dimensional datasets , 2010 .

[62]  Ashley Prater,et al.  Classification via tensor decompositions of echo state networks , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[63]  Peter Lindstrom,et al.  TTHRESH: Tensor Compression for Multidimensional Visual Data , 2018, IEEE Transactions on Visualization and Computer Graphics.

[64]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[65]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[66]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[67]  Snigdhansu Chatterjee,et al.  Procrustes Problems , 2005, Technometrics.

[68]  Pierre Comon,et al.  Tensors : A brief introduction , 2014, IEEE Signal Processing Magazine.

[69]  Panos P. Markopoulos,et al.  Novel Algorithms for Exact and Efficient L1-NORM-BASED Tucker2 Decomposition , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).