Compressed Randomized UTV Decompositions for Low-Rank Matrix Approximations

Low-rank matrix approximations play a fundamental role in numerical linear algebra and signal processing applications. This paper introduces a novel rank-revealing matrix decomposition algorithm termed compressed randomized UTV (CoR-UTV) decomposition along with a CoR-UTV variant aided by the power method technique. CoR-UTV is primarily developed to compute an approximation to a low-rank input matrix by making use of random sampling schemes. Given a large and dense matrix of size <inline-formula><tex-math notation="LaTeX">$m\times n$</tex-math></inline-formula> with numerical rank <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX">$k \ll \text{min} \lbrace m,n\rbrace$</tex-math></inline-formula>, CoR-UTV requires a few passes over the data, and runs in <inline-formula><tex-math notation="LaTeX">$O(mnk)$</tex-math></inline-formula> floating-point operations. Furthermore, CoR-UTV can exploit modern computational platforms and, consequently, can be optimized for maximum efficiency. CoR-UTV is simple and accurate, and outperforms reported alternative methods in terms of efficiency and accuracy. Simulations with synthetic data as well as real data in image reconstruction and robust principal component analysis applications support our claims.

[1]  Russ B. Altman,et al.  Missing value estimation methods for DNA microarrays , 2001, Bioinform..

[2]  T. Chan Rank revealing QR factorizations , 1987 .

[3]  Petros Drineas,et al.  FAST MONTE CARLO ALGORITHMS FOR MATRICES II: COMPUTING A LOW-RANK APPROXIMATION TO A MATRIX∗ , 2004 .

[4]  Jed A. Duersch,et al.  Randomized QR with Column Pivoting , 2015, SIAM J. Sci. Comput..

[5]  J. E. Jackson A User's Guide to Principal Components , 1991 .

[6]  James Demmel,et al.  Communication Avoiding Rank Revealing QR Factorization with Column Pivoting , 2015, SIAM J. Matrix Anal. Appl..

[7]  P. Hansen Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[8]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Maboud Farzaneh Kaloorazi,et al.  Subspace-Orbit Randomized Decomposition for Low-Rank Matrix Approximations , 2018, IEEE Transactions on Signal Processing.

[10]  Fatih Murat Porikli,et al.  CDnet 2014: An Expanded Change Detection Benchmark Dataset , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[11]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[12]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[13]  Qi Tian,et al.  Statistical modeling of complex backgrounds for foreground object detection , 2004, IEEE Transactions on Image Processing.

[14]  Emmanuel J. Candès,et al.  Robust Subspace Clustering , 2013, ArXiv.

[15]  Rodrigo C. de Lamare,et al.  Anomaly detection in IP networks based on randomized subspace methods , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Jack J. Dongarra,et al.  With Extreme Computing, the Rules Have Changed , 2017, Computing in Science & Engineering.

[17]  Thierry Bouwmans,et al.  Robust PCA via Principal Component Pursuit: A review for a comparative evaluation in video surveillance , 2014, Comput. Vis. Image Underst..

[18]  S. Joe Qin,et al.  Subspace approach to multidimensional fault identification and reconstruction , 1998 .

[19]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[20]  Ming Gu,et al.  Efficient Algorithms for Computing a Strong Rank-Revealing QR Factorization , 1996, SIAM J. Sci. Comput..

[21]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[22]  George Atia,et al.  High Dimensional Low Rank Plus Sparse Matrix Decomposition , 2015, IEEE Transactions on Signal Processing.

[23]  George Atia,et al.  Coherence Pursuit: Fast, Simple, and Robust Principal Component Analysis , 2016, IEEE Transactions on Signal Processing.

[24]  Tae-Hyun Oh,et al.  Fast Randomized Singular Value Thresholding for Low-Rank Optimization , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Paul Tseng,et al.  Hankel Matrix Rank Minimization with Applications to System Identification and Realization , 2013, SIAM J. Matrix Anal. Appl..

[26]  D. Calvetti,et al.  AN IMPLICITLY RESTARTED LANCZOS METHOD FOR LARGE SYMMETRIC EIGENVALUE PROBLEMS , 1994 .

[27]  Huy L. Nguyen,et al.  OSNAP: Faster Numerical Linear Algebra Algorithms via Sparser Subspace Embeddings , 2012, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[28]  G. W. Stewart,et al.  Matrix Algorithms: Volume 1, Basic Decompositions , 1998 .

[29]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[30]  G. Stewart Updating a Rank-Revealing ULV Decomposition , 1993, SIAM J. Matrix Anal. Appl..

[31]  R. C. Thompson Principal submatrices IX: Interlacing inequalities for singular values of submatrices , 1972 .

[32]  Santosh S. Vempala,et al.  Adaptive Sampling and Fast Low-Rank Matrix Approximation , 2006, APPROX-RANDOM.

[33]  J. Edward Jackson,et al.  A User's Guide to Principal Components. , 1991 .

[34]  Tamás Sarlós,et al.  Improved Approximation Algorithms for Large Matrices via Random Projections , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[35]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[36]  Rodrigo C. de Lamare,et al.  Low-rank and sparse matrix recovery based on a randomized rank-revealing decomposition , 2017, 2017 22nd International Conference on Digital Signal Processing (DSP).

[37]  Noga Alon,et al.  Generalization Error Bounds for Collaborative Prediction with Low-Rank Matrices , 2004, NIPS.

[38]  Xiaoming Yuan,et al.  Sparse and low-rank matrix decomposition via alternating direction method , 2013 .

[39]  A. Willsky,et al.  Latent variable graphical model selection via convex optimization , 2010 .

[40]  David P. Woodruff,et al.  Low rank approximation and regression in input sparsity time , 2013, STOC '13.

[41]  Mark Tygert,et al.  A Randomized Algorithm for Principal Component Analysis , 2008, SIAM J. Matrix Anal. Appl..

[42]  Per Christian Hansen,et al.  Low-rank revealing UTV decompositions , 1997, Numerical Algorithms.

[43]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[44]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[45]  Siam J. Sci,et al.  SUBSPACE ITERATION RANDOMIZATION AND SINGULAR VALUE PROBLEMS , 2015 .

[46]  R. D. Lamare,et al.  Adaptive Reduced-Rank Processing Based on Joint and Iterative Interpolation, Decimation, and Filtering , 2009, IEEE Transactions on Signal Processing.

[47]  Morteza Mardani,et al.  Decentralized Sparsity-Regularized Rank Minimization: Algorithms and Applications , 2012, IEEE Transactions on Signal Processing.

[48]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[49]  Alan M. Frieze,et al.  Fast Monte-Carlo algorithms for finding low-rank approximations , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[50]  Robert A. van de Geijn,et al.  Householder QR Factorization With Randomization for Column Pivoting (HQRRP) , 2015, SIAM J. Sci. Comput..

[51]  Volkan Cevher,et al.  Practical Sketching Algorithms for Low-Rank Matrix Approximation , 2016, SIAM J. Matrix Anal. Appl..

[52]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[53]  Stephen Becker,et al.  Quantum state tomography via compressed sensing. , 2009, Physical review letters.

[54]  Sudeep Sarkar,et al.  An evaluation of face and ear biometrics , 2002, Object recognition supported by user interaction for service robots.

[55]  James Demmel,et al.  Communication-optimal Parallel and Sequential QR and LU Factorizations , 2008, SIAM J. Sci. Comput..

[56]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[57]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[59]  Mark Rudelson,et al.  Sampling from large matrices: An approach through geometric functional analysis , 2005, JACM.

[60]  L. Mirsky SYMMETRIC GAUGE FUNCTIONS AND UNITARILY INVARIANT NORMS , 1960 .

[61]  Per Christian Hansen,et al.  UTV Tools: Matlab templates for rank-revealing UTV decompositions , 1999, Numerical Algorithms.

[62]  G. W. Stewart,et al.  An updating algorithm for subspace tracking , 1992, IEEE Trans. Signal Process..