Online Rank-Revealing Block-Term Tensor Decomposition

The so-called block-term decomposition (BTD) tensor model, especially in its rank-(Lr, Lr, 1) version, has been recently receiving increasing attention due to its enhanced ability of representing systems and signals that are composed of block components of rank higher than one, a scenario encountered in numerous and diverse applications. Its uniqueness and approximation have thus been thoroughly studied. The challenging problem of estimating the BTD model structure, namely the number of block terms (rank) and their individual (block) ranks, is of crucial importance in practice and has only recently started to attract significant attention. In data-streaming scenarios and/or big data applications, where the tensor dimension in one of its modes grows in time or can only be processed incrementally, it is essential to be able to perform model selection and computation in a recursive (incremental/online) manner. To date there is only one such work in the literature concerning the (general rank(L,M,N)) BTD model, which proposes an incremental method, however with the BTD rank and block ranks assumed to be a-priori known and time invariant. In this preprint, a novel approach to rank-(Lr, Lr, 1) BTD model selection and tracking is proposed, based on the idea of imposing column sparsity jointly on the factors and estimating the ranks as the numbers of factor columns of nonnegligible magnitude. An online method of the alternating iteratively reweighted least squares (IRLS) type is developed and shown to be computationally efficient and fast converging, also allowing the model ranks to change in time. Its time and memory efficiency are evaluated and favorably compared with those of the batch approach. Simulation results are reported that demonstrate the effectiveness of the proposed scheme in both selecting and tracking the correct BTD model.

[1]  Seid Miad Zandavi,et al.  Online Tensor Decomposition with optimized Stochastic Gradient Descent: an Application in Structural Damage Identification , 2020, 2020 IEEE Symposium Series on Computational Intelligence (SSCI).

[2]  Shuo Zhou.  On dynamic tensor decompositions , 2019 .

[3]  Paris V. Giampouras,et al.  Online Reweighted Least Squares Robust PCA , 2020, IEEE Signal Processing Letters.

[4]  Hanbaek Lyu.  Convergence of block coordinate descent with diminishing radius for nonconvex optimization , 2020 .

[5]  Paris V. Giampouras,et al.  Rank-Revealing Block-Term Decomposition for Tensor Completion , 2021, ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Hailong Sun,et al.  An Incremental Tensor Factorization Approach for Web Service Recommendation , 2014, 2014 IEEE International Conference on Data Mining Workshop.

[7]  Paris V. Giampouras,et al.  Alternating Iteratively Reweighted Least Squares Minimization for Low-Rank Matrix Factorization , 2019, IEEE Transactions on Signal Processing.

[8]  Elizaveta Rebrova,et al.  On Nonnegative Matrix and Tensor Decompositions for COVID-19 Twitter Dynamics , 2020, ArXiv.

[9]  Emiliano Dall'Anese,et al.  Optimization and Learning With Information Streams: Time-varying algorithms and applications , 2020, IEEE Signal Processing Magazine.

[10]  Michiel Vandecappelle,et al.  Low Multilinear Rank Updating , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[11]  Kun Xie,et al.  Online MECG Compression based on Incremental Tensor Decomposition for Wearable Devices. , 2020, IEEE journal of biomedical and health informatics.

[12]  Bahador Makkiabadi,et al.  Evaluation of adaptive parafac alogorithms for tracking of simulated moving brain sources , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[13]  Hao Zhang,et al.  Incremental Factorization of Big Time Series Data with Blind Factor Approximation , 2019, IEEE Transactions on Knowledge and Data Engineering.

[14]  Soon Ki Jung,et al.  Online Stochastic Tensor Decomposition for Background Subtraction in Multispectral Video Sequences , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[15]  Yang Guo,et al.  Low-Rank Tucker Approximation of a Tensor From Streaming Data , 2019, SIAM J. Math. Data Sci..

[16]  Seid Miad Zandavi,et al.  NeCPD: An Online Tensor Decomposition with Optimal Stochastic Gradient Descent , 2020, ArXiv.

[17]  Georgios B. Giannakis,et al.  Time-Varying Convex Optimization: Time-Structured Algorithms and Applications , 2020, Proceedings of the IEEE.

[18]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[19]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[20]  Stephen Becker,et al.  Low-Rank Tucker Decomposition of Large Tensors Using TensorSketch , 2018, NeurIPS.

[21]  Philip S. Yu,et al.  Incremental tensor analysis: Theory and applications , 2008, TKDD.

[22]  Shun-Tian Lou,et al.  Adaptive Blind Channel Estimation for MIMO-OFDM Systems Based on PARAFAC , 2020, Wirel. Commun. Mob. Comput..

[23]  Panagiotis Symeonidis,et al.  Matrix and Tensor Factorization Techniques for Recommender Systems , 2017, SpringerBriefs in Computer Science.

[24]  Xiaoqin Zhang,et al.  Incremental Tensor Subspace Learning and Its Applications to Foreground Segmentation and Tracking , 2011, International Journal of Computer Vision.

[25]  Hiroyuki Kasai,et al.  Fast online low-rank tensor subspace tracking by CP decomposition using recursive least squares from incomplete observations , 2017, Neurocomputing.

[26]  Xuelong Li,et al.  Incremental learning of weighted tensor subspace for visual tracking , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[27]  Hong Chen,et al.  Location-Based Recommendation Using Incremental Tensor Factorization Model , 2014, ADMA.

[28]  James Caverlee,et al.  Multi-Aspect Streaming Tensor Completion , 2017, KDD.

[29]  Hongshan Zhao,et al.  Power Distribution System Stream Data Compression Based on Incremental Tensor Decomposition , 2020, IEEE Transactions on Industrial Informatics.

[30]  Jiliu Zhou,et al.  Incremental Tensor Principal Component Analysis for Handwritten Digit Recognition , 2014 .

[31]  Nico Vervliet,et al.  Nonlinear least squares updating of the canonical polyadic decomposition , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[32]  Hadi Fanaee-T,et al.  Tensor decomposition for analysing time-evolving social networks: an overview , 2020, Artificial Intelligence Review.

[33]  Jun Zhou,et al.  Hyperspectral Restoration via $L_0$ Gradient Regularized Low-Rank Tensor Factorization , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[35]  Evangelos E. Papalexakis,et al.  Identifying and Alleviating Concept Drift in Streaming Tensor Decomposition , 2018, ECML/PKDD.

[36]  Sergios Theodoridis,et al.  Blind fMRI source unmixing via higher-order tensor decompositions , 2019, Journal of Neuroscience Methods.

[37]  Miguel Sozinho Ramalho.  Incremental Multi-Dimensional Recommender Systems: Co-Factorization vs Tensors , 2019, ORSUM@RecSys.

[38]  Evangelos E. Papalexakis,et al.  OnlineBTD: Streaming Algorithms to Track the Block Term Decomposition of Large Tensors , 2020, 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA).

[39]  Fei Wang,et al.  eOTD: An Efficient Online Tucker Decomposition for Higher Order Tensors , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[40]  Evangelos E. Papalexakis,et al.  SPADE: Streaming PARAFAC2 DEcomposition for Large Datasets , 2020, SDM.

[41]  Nikos D. Sidiropoulos,et al.  Adaptive Algorithms to Track the PARAFAC Decomposition of a Third-Order Tensor , 2009, IEEE Transactions on Signal Processing.

[42]  Anima Anandkumar,et al.  Online tensor methods for learning latent variable models , 2013, J. Mach. Learn. Res..

[43]  Paris V. Giampouras,et al.  Block-Term Tensor Decomposition: Model Selection and Computation , 2021, IEEE Journal of Selected Topics in Signal Processing.

[44]  Karim Abed-Meraim,et al.  Adaptive Algorithms for Tracking Tensor-Train Decomposition of Streaming Tensors , 2020, 2020 28th European Signal Processing Conference (EUSIPCO).

[45]  Philip S. Yu,et al.  Outlier-Robust Multi-Aspect Streaming Tensor Completion and Factorization , 2019, IJCAI.

[46]  Yan Wang,et al.  A Dimensionality Reduction Algorithm for Unstructured Campus Big Data Fusion , 2021, Symmetry.

[47]  Karim Abed-Meraim,et al.  Second-order optimization based adaptive PARAFAC decomposition of three-way tensors , 2017, Digit. Signal Process..

[48]  Luis Javier Segura,et al.  Online droplet anomaly detection from streaming videos in inkjet printing , 2021 .

[49]  Yuan Yan Tang,et al.  Matrix-Vector Nonnegative Tensor Factorization for Blind Unmixing of Hyperspectral Imagery , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Morteza Mardani,et al.  Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors , 2014, IEEE Transactions on Signal Processing.

[51]  Rose Yu,et al.  Accelerated Online Low Rank Tensor Learning for Multivariate Spatiotemporal Streams , 2015, ICML.

[52]  Georgios B. Giannakis,et al.  Tensor Decompositions for Identifying Directed Graph Topologies and Tracking Dynamic Networks , 2016, IEEE Transactions on Signal Processing.

[53]  Jarvis D. Haupt,et al.  Provable Online CP/PARAFAC Decomposition of a Structured Tensor via Dictionary Learning , 2020, NeurIPS.

[54]  Truong Minh-Chinh,et al.  Adaptive PARAFAC decomposition for third-order tensor completion , 2016, 2016 IEEE Sixth International Conference on Communications and Electronics (ICCE).

[55]  Xiaowei Yang,et al.  Randomized online CP decomposition , 2018, 2018 Tenth International Conference on Advanced Computational Intelligence (ICACI).

[56]  K. Selçuk Candan,et al.  BICP: Block-Incremental CP Decomposition with Update Sensitive Refinement , 2016, CIKM.

[57]  Shandian Zhe,et al.  Probabilistic Streaming Tensor Decomposition , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[58]  Alain Rakotomamonjy,et al.  Online multimodal dictionary learning , 2019, Neurocomputing.

[59]  James Bailey,et al.  Online CP Decomposition for Sparse Tensors , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[60]  Nguyen Lu Dang Khoa,et al.  Online Data Fusion Using Incremental Tensor Learning , 2019, PAKDD.

[61]  Anima Anandkumar,et al.  Online and Differentially-Private Tensor Decomposition , 2016, NIPS.

[62]  Othmar Koch,et al.  Dynamical Tensor Approximation , 2010, SIAM J. Matrix Anal. Appl..

[63]  Laurence T. Yang,et al.  A Tensor-Based Approach for Big Data Representation and Dimensionality Reduction , 2014, IEEE Transactions on Emerging Topics in Computing.

[64]  Hadi Fanaee-T,et al.  Multi-aspect-streaming tensor analysis , 2015, Knowl. Based Syst..

[65]  Xiaoqin Zhang,et al.  Robust Visual Tracking Based on Incremental Tensor Subspace Learning , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[66]  Lieven De Lathauwer,et al.  Block Component Analysis, a New Concept for Blind Source Separation , 2012, LVA/ICA.

[67]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[68]  Zhi-Quan Luo,et al.  A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data: With applications in machine learning and signal processing , 2015, IEEE Signal Processing Magazine.

[69]  Nazanin Rahnavard,et al.  Primary User Localization and Online Radio Cartography via Structured Tensor Decomposition , 2019, ArXiv.

[70]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[71]  Adel Hafiane,et al.  A Fast Randomized Adaptive CP Decomposition For Streaming Tensors , 2021, ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[72]  Dimitris G. Chachlakis,et al.  Dynamic L1-Norm Tucker Tensor Decomposition , 2020, IEEE Journal of Selected Topics in Signal Processing.

[73]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[74]  Sergios Theodoridis,et al.  Machine Learning: A Bayesian and Optimization Perspective , 2015 .

[75]  Thierry Bouwmans,et al.  Incremental and Multi-feature Tensor Subspace Learning Applied for Background Modeling and Subtraction , 2014, ICIAR.

[76]  Manish Gupta,et al.  Inductive Framework for Multi-Aspect Streaming Tensor Completion with Side Information , 2018, CIKM.

[77]  Evangelos E. Papalexakis,et al.  SamBaTen: Sampling-based Batch Incremental Tensor Decomposition , 2017, SDM.

[78]  James Bailey,et al.  Accelerating Online CP Decompositions for Higher Order Tensors , 2016, KDD.

[79]  Evangelos E. Papalexakis,et al.  OCTEN: Online Compression-Based Tensor Decomposition , 2018, 2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[80]  Nikos D. Sidiropoulos,et al.  Streaming Tensor Factorization for Infinite Data Sources , 2018, SDM.

[81]  Paris V. Giampouras,et al.  Block-Term Tensor Decomposition: Model Selection and Computation , 2020, 2020 28th European Signal Processing Conference (EUSIPCO).