Best Pair Formulation & Accelerated Scheme for Non-Convex Principal Component Pursuit

Given two disjoint sets, the best pair problem aims to find a point in one set and another point in the other set with minimal distance between them. In this paper, we formulate the classical robust principal component analysis (RPCA) problem as a best pair problem and design an accelerated proximal gradient algorithm to solve it. We prove that the method enjoys global convergence with a local linear rate. Our extensive numerical experiments on both real and synthetic data sets suggest that our proposed algorithm outperforms relevant baseline alggorithms in the literature.

[1]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[2]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[3]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[4]  Constantine Caramanis,et al.  Robust Matrix Completion and Corrupted Columns , 2011, ICML.

[5]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[6]  Peilin Liu,et al.  A Survey on Nonconvex Regularization-Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning , 2018, IEEE Access.

[7]  Pierre-Antoine Absil,et al.  Robust Low-Rank Matrix Completion by Riemannian Optimization , 2016, SIAM J. Sci. Comput..

[8]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[10]  Lucia Maddalena,et al.  Scene background initialization: A taxonomy , 2017, Pattern Recognit. Lett..

[11]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[12]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[13]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[14]  Arvind Ganesh,et al.  Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix , 2009 .

[15]  In-So Kweon,et al.  Partial Sum Minimization of Singular Values in Robust PCA: Algorithm and Applications , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  A. Tsybakov,et al.  Robust matrix completion , 2014, Probability Theory and Related Fields.

[17]  Aritra Dutta,et al.  A Batch-Incremental Video Background Estimation Model Using Weighted Low-Rank Approximation of Matrices , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[18]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[19]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.

[20]  Fei Wen,et al.  Nonconvex Regularized Robust PCA Using the Proximal Block Coordinate Descent Algorithm , 2019, IEEE Transactions on Signal Processing.

[21]  Jingwei Liang,et al.  Convergence Rates of First-Order Operator Splitting Methods. (Taux de convergence des méthodes du premier ordre d'éclatement d'opérateurs) , 2016 .

[22]  Aswin C. Sankaranarayanan,et al.  SpaRCS: Recovering low-rank and sparse matrices from compressive measurements , 2011, NIPS.

[23]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Stephen P. Boyd,et al.  Generalized Low Rank Models , 2014, Found. Trends Mach. Learn..

[25]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[26]  J. Bolte,et al.  Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity , 2009 .

[27]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[28]  Xiaoming Yuan,et al.  Sparse and low-rank matrix decomposition via alternating direction method , 2013 .

[29]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[30]  Gilad Lerman,et al.  Robust Stochastic Principal Component Analysis , 2014, AISTATS.

[31]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Mathematical Programming Computation.

[32]  Dacheng Tao,et al.  GoDec: Randomized Lowrank & Sparse Matrix Decomposition in Noisy Case , 2011, ICML.

[33]  Yongli Wang,et al.  Robust PCA Using Matrix Factorization for Background/Foreground Separation , 2018, IEEE Access.

[34]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[35]  Aritra Dutta,et al.  A Nonconvex Projection Method for Robust PCA , 2018, AAAI.

[36]  Aritra Dutta,et al.  Weighted Low Rank Approximation for Background Estimation Problems , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[37]  Prateek Jain,et al.  Non-convex Robust PCA , 2014, NIPS.

[38]  Thierry Bouwmans,et al.  Robust PCA via Principal Component Pursuit: A review for a comparative evaluation in video surveillance , 2014, Comput. Vis. Image Underst..

[39]  Prateek Jain,et al.  Thresholding Based Outlier Robust PCA , 2017, COLT.

[40]  Soon Ki Jung,et al.  Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset , 2015, Comput. Sci. Rev..

[41]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[42]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[43]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[44]  Teng Zhang,et al.  Robust PCA by Manifold Optimization , 2017, J. Mach. Learn. Res..

[45]  Benjamin Höferlin,et al.  Evaluation of background subtraction techniques for video surveillance , 2011, CVPR 2011.

[46]  Sham M. Kakade,et al.  Robust Matrix Decomposition With Sparse Corruptions , 2011, IEEE Transactions on Information Theory.

[47]  Mohamed-Jalal Fadili,et al.  A Multi-step Inertial Forward-Backward Splitting Method for Non-convex Optimization , 2016, NIPS.

[48]  Xin Li,et al.  Weighted Low-Rank Approximation of Matrices and Background Modeling , 2018, ArXiv.

[49]  Aritra Dutta,et al.  Online and Batch Supervised Background Estimation Via L1 Regression , 2017, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[50]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[51]  Mohamed-Jalal Fadili,et al.  Local Linear Convergence of Forward-Backward under Partial Smoothness , 2014, NIPS.

[52]  Aritra Dutta,et al.  A Fast Weighted SVT Algorithm , 2018, 2018 5th International Conference on Systems and Informatics (ICSAI).

[53]  Dmitriy Drusvyatskiy,et al.  Optimality, identifiability, and sensitivity , 2012, Math. Program..

[54]  Prateek Jain,et al.  Fast Exact Matrix Completion with Finite Samples , 2014, COLT.

[55]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[56]  Xiaodong Li,et al.  Compressed Sensing and Matrix Completion with Constant Proportion of Corruptions , 2011, Constructive Approximation.

[57]  Xingzhao Liu,et al.  Robust Matrix Completion via Alternating Projection , 2017, IEEE Signal Processing Letters.

[58]  Jakub Marecek,et al.  Matrix completion under interval uncertainty , 2017, Eur. J. Oper. Res..

[59]  Feiping Nie,et al.  Robust Matrix Completion via Joint Schatten p-Norm and lp-Norm Minimization , 2012, 2012 IEEE 12th International Conference on Data Mining.

[60]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[61]  Thierry Bouwmans,et al.  Traditional and recent approaches in background modeling for foreground detection: An overview , 2014, Comput. Sci. Rev..

[62]  Prateek Jain,et al.  Nearly Optimal Robust Matrix Completion , 2016, ICML.

[63]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[64]  Constantine Caramanis,et al.  Fast Algorithms for Robust PCA via Gradient Descent , 2016, NIPS.

[65]  Adrian S. Lewis,et al.  Partial Smoothness, Tilt Stability, and Generalized Hessians , 2013, SIAM J. Optim..

[66]  Anton van den Hengel,et al.  Semidefinite Programming , 2014, Computer Vision, A Reference Guide.