Multilinear Low-Rank Tensors on Graphs & Applications

We propose a new framework for the analysis of low-rank tensors which lies at the intersection of spectral graph theory and signal processing. As a first step, we present a new graph based low-rank decomposition which approximates the classical low-rank SVD for matrices and multi-linear SVD for tensors. Then, building on this novel decomposition we construct a general class of convex optimization problems for approximately solving low-rank tensor inverse problems, such as tensor Robust PCA. The whole framework is named as 'Multilinear Low-rank tensors on Graphs (MLRTG)'. Our theoretical analysis shows: 1) MLRTG stands on the notion of approximate stationarity of multi-dimensional signals on graphs and 2) the approximation error depends on the eigen gaps of the graphs. We demonstrate applications for a wide variety of 4 artificial and 12 real tensor datasets, such as EEG, FMRI, BCI, surveillance videos and hyperspectral images. Generalization of the tensor concepts to non-euclidean domain, orders of magnitude speed-up, low-memory requirement and significantly enhanced performance at low SNR are the key aspects of our framework.

[1]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[2]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[3]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Pierre Vandergheynst,et al.  Stationary Signal Processing on Graphs , 2016, IEEE Transactions on Signal Processing.

[5]  Xavier Bresson,et al.  Robust Principal Component Analysis on Graphs , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  Pierre Vandergheynst,et al.  Compressive PCA for Low-Rank Matrices on Graphs , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[7]  Charalampos E. Tsourakakis.  MACH: Fast Randomized Tensor Decompositions , 2009, SDM.

[8]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[9]  Nathanael Perraudin,et al.  Fast Robust PCA on Graphs , 2015, IEEE Journal of Selected Topics in Signal Processing.

[10]  Anima Anandkumar,et al.  Online tensor methods for learning latent variable models , 2013, J. Mach. Learn. Res..

[11]  Andrzej Cichocki,et al.  Fast Alternating LS Algorithms for High Order CANDECOMP/PARAFAC Tensor Factorizations , 2013, IEEE Transactions on Signal Processing.

[12]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[13]  Pradeep Ravikumar,et al.  Collaborative Filtering with Graph Information: Consistency and Scalable Methods , 2015, NIPS.

[14]  Christos Faloutsos,et al.  GigaTensor: scaling tensor analysis up by 100 times - algorithms and discoveries , 2012, KDD.

[15]  Anima Anandkumar,et al.  Fast Detection of Overlapping Communities via Online Tensor Methods on GPUs , 2013, ArXiv.

[16]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[17]  Xiaogang Wang,et al.  Convolutional neural networks with low-rank regularization , 2015, ICLR.

[18]  Yu-Jin Zhang,et al.  Neighborhood Preserving Non-negative Tensor Factorization for image representation , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Ryota Tomioka,et al.  On the extension of trace norm to tensors , 2010 .

[20]  Furong Huang.  Latent Dirichlet Allocation via Tensor Factorization , 2014 .

[21]  Sujay Sanghavi,et al.  A New Sampling Technique for Tensors , 2015, ArXiv.

[22]  Inderjit S. Dhillon,et al.  Multi-Scale Spectral Decomposition of Massive Graphs , 2014, NIPS.

[23]  Alexander J. Smola,et al.  Fast and Guaranteed Tensor Decomposition via Sketching , 2015, NIPS.

[24]  Pierre Vandergheynst,et al.  UNLocBoX A matlab convex optimization toolbox using proximal splitting methods , 2014, ArXiv.

[25]  Pierre Vandergheynst,et al.  GSPBOX: A toolbox for signal processing on graphs , 2014, ArXiv.

[26]  Mohamed Cheriet,et al.  Large Margin Low Rank Tensor Analysis , 2013, Neural Computation.

[27]  Bart Vandereycken,et al.  Low-rank tensor completion by Riemannian optimization , 2014 .

[28]  Lionel Martin,et al.  Fast Eigenspace Approximation using Random Signals , 2016, ArXiv.

[29]  Chun Chen,et al.  Image representation using Laplacian regularized nonnegative tensor factorization , 2011, Pattern Recognit..

[30]  John Wright,et al.  Provable Models for Robust Low-Rank Tensor Completion , 2015 .

[31]  David G. Lowe,et al.  Scalable Nearest Neighbor Algorithms for High Dimensional Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  J. H. Choi,et al.  DFacTo: Distributed Factorization of Tensors , 2014, NIPS.

[33]  Ryota Tomioka,et al.  Estimation of low-rank tensors via convex optimization , 2010, 1010.0789.

[34]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[35]  Donald Goldfarb,et al.  Robust Low-Rank Tensor Recovery: Models and Algorithms , 2013, SIAM J. Matrix Anal. Appl..

[36]  C. D. Silva.  Hierarchical Tucker Tensor Optimization-Applications to Tensor Completion , 2013 .