Dynamic L1-Norm Tucker Tensor Decomposition

Tucker decomposition is a standard method for processing multi-way (tensor) measurements and finds many applications in machine learning and data mining, among other fields. When tensor measurements arrive in a streaming fashion or are too many to jointly decompose, incremental Tucker analysis is preferred. In addition, dynamic basis adaptation is desired when the nominal data subspaces change. At the same time, it has been documented that outliers in the data can significantly compromise the performance of existing methods for dynamic Tucker analysis. In this work, we present Dynamic L1-Tucker: an algorithm for dynamic and outlier-resistant Tucker analysis of tensor data. Our experimental studies on both real and synthetic datasets corroborate that the proposed method (i) attains high basis estimation performance, (ii) identifies/rejects outliers, and (iii) adapts to nominal subspace changes. Index Terms – Data analysis, L1-norm, outliers, robust, tensors, Tucker decomposition.

[1]  A. Stechow,et al.  Decomposition , 1902, The Indian medical gazette.

[2]  Josef Schmee,et al.  Outliers in Statistical Data (2nd ed.) , 1986 .

[3]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[4]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[5]  R. Viertl.  On the Future of Data Analysis , 2002 .

[6]  Gert Cauwenberghs,et al.  SVM incremental learning, adaptation and optimization , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[7]  Jian Yang,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Xuelong Li,et al.  Supervised tensor learning , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[9]  Jimeng Sun,et al.  Beyond streams and graphs: dynamic tensor analysis , 2006, KDD '06.

[10]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[11]  Xiaoqin Zhang,et al.  Robust Visual Tracking Based on Incremental Tensor Subspace Learning , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[12]  Philip S. Yu,et al.  Incremental tensor analysis: Theory and applications , 2008, TKDD.

[13]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[14]  Xuelong Li,et al.  Robust Tensor Analysis With L1-Norm , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Xiaoqin Zhang,et al.  Incremental Tensor Subspace Learning and Its Applications to Foreground Segmentation and Tracking , 2011, International Journal of Computer Vision.

[16]  Erik G. Larsson,et al.  EVD-based channel estimation in multicell multiuser MIMO systems with very large antenna arrays , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Rasmus Pagh,et al.  Compressed matrix multiplication , 2011, ITCS '12.

[18]  Thierry Bouwmans,et al.  Incremental and Multi-feature Tensor Subspace Learning Applied for Background Modeling and Subtraction , 2014, ICIAR.

[19]  Donald Goldfarb,et al.  Robust Low-Rank Tensor Recovery: Models and Algorithms , 2013, SIAM J. Matrix Anal. Appl..

[20]  Panos P. Markopoulos,et al.  Optimal Algorithms for L1-subspace Signal Processing , 2014, IEEE Transactions on Signal Processing.

[21]  Leo Marco,et al.  Accuracy on a subset of the Extended Yale Face Database B. , 2014 .

[22]  Nikos D. Sidiropoulos,et al.  Joint Tensor Factorization and Outlying Slab Suppression With Applications , 2015, IEEE Transactions on Signal Processing.

[23]  Rose Yu,et al.  Accelerated Online Low Rank Tensor Learning for Multivariate Spatiotemporal Streams , 2015, ICML.

[24]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[25]  Dongdai Lin,et al.  Robust Face Clustering Via Tensor Decomposition , 2015, IEEE Transactions on Cybernetics.

[26]  Tahina Ramananandro,et al.  Accelerated low-rank updates to tensor decompositions , 2016, 2016 IEEE High Performance Extreme Computing Conference (HPEC).

[27]  Piya Pal,et al.  On canonical polyadic decomposition of overcomplete tensors of arbitrary even order , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[28]  Nikos D. Sidiropoulos,et al.  Tensors for Data Mining and Data Fusion , 2016, ACM Trans. Intell. Syst. Technol..

[29]  Panos P. Markopoulos,et al.  Efficient L1-Norm Principal-Component Analysis via Bit Flipping , 2016, IEEE Transactions on Signal Processing.

[30]  Martin Haardt,et al.  HOSVD-Based Denoising for Improved Channel Prediction of Weak Massive MIMO Channels , 2017, 2017 IEEE 85th Vehicular Technology Conference (VTC Spring).

[31]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[32]  Matthieu Cord,et al.  MUTAN: Multimodal Tucker Fusion for Visual Question Answering , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Nikos D. Sidiropoulos,et al.  Streaming Tensor Factorization for Infinite Data Sources , 2018, SDM.

[34]  Panos P. Markopoulos,et al.  Adaptive L1-Norm Principal-Component Analysis With Online Outlier Rejection , 2018, IEEE Journal of Selected Topics in Signal Processing.

[35]  Shandian Zhe,et al.  Probabilistic Streaming Tensor Decomposition , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[36]  Panos P. Markopoulos,et al.  Robust decomposition of 3-way tensors based on L1-norm , 2018, Commercial + Scientific Sensing and Imaging.

[37]  Panos P. Markopoulos,et al.  Novel Algorithms for Exact and Efficient L1-NORM-BASED Tucker2 Decomposition , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[38]  Panos P. Markopoulos,et al.  L1-NORM HIGHER-ORDER SINGULAR-VALUE DECOMPOSITION , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[39]  Stephen Becker,et al.  Low-Rank Tucker Decomposition of Large Tensors Using TensorSketch , 2018, NeurIPS.

[40]  Panos P. Markopoulos,et al.  The Exact Solution to Rank-1 L1-Norm TUCKER2 Decomposition , 2017, IEEE Signal Processing Letters.

[41]  Panos P. Markopoulos,et al.  Iteratively Re-weighted L1-PCA of Tensor Data , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[42]  Panos P. Markopoulos,et al.  Options for multimodal classification based on L1-Tucker decomposition , 2019, Big Data: Learning, Analytics, and Applications.

[43]  Luming Zhang,et al.  Online Robust Low-Rank Tensor Modeling for Streaming Data Analysis , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Philip S. Yu,et al.  Outlier-Robust Multi-Aspect Streaming Tensor Completion and Factorization , 2019, IJCAI.

[45]  Panos P. Markopoulos,et al.  L1-Norm Tucker Tensor Decomposition , 2019, IEEE Access.

[46]  Rafael T. de Sousa,et al.  Tensor-Based Channel Estimation for Massive MIMO-OFDM Systems , 2019, IEEE Access.

[47]  Peter Lindstrom,et al.  TTHRESH: Tensor Compression for Multidimensional Visual Data , 2018, IEEE Transactions on Visualization and Computer Graphics.

[48]  Panos P. Markopoulos,et al.  L1-Norm Higher-Order Orthogonal Iterations for Robust Tensor Analysis , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[49]  Yang Guo,et al.  Low-Rank Tucker Approximation of a Tensor From Streaming Data , 2019, SIAM J. Math. Data Sci..