Robust Elastic-Net Subspace Representation

Recently, finding the low-dimensional structure of high-dimensional data has gained much attention. Given a set of data points sampled from a single subspace or a union of subspaces, the goal is to learn or capture the underlying subspace structure of the data set. In this paper, we propose elastic-net subspace representation, a new subspace representation framework using elastic-net regularization of singular values. Due to the strong convexity enforced by elastic-net, the proposed method is more stable and robust in the presence of heavy corruptions compared with existing lasso-type rank minimization approaches. For discovering a single low-dimensional subspace, we propose a computationally efficient low-rank factorization algorithm, called FactEN, using a property of the nuclear norm and the augmented Lagrangian method. Then, ClustEN is proposed to handle the general case, in which the data samples are drawn from a union of multiple subspaces, for joint subspace clustering and estimation. The proposed algorithms are applied to a number of subspace representation problems to evaluate the robustness and efficiency under various noisy conditions, and experimental results show the benefits of the proposed method compared with existing methods.

[1]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[2]  Robert Tibshirani,et al.  Spectral Regularization Algorithms for Learning Large Incomplete Matrices , 2010, J. Mach. Learn. Res..

[3]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[4]  J. Ferryman,et al.  PETS2009: Dataset and challenge , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[5]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[6]  Cun-Hui Zhang,et al.  Calibrated Elastic Regularization in Matrix Completion , 2012, NIPS.

[7]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[8]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[9]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[10]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[11]  Cédric Herzet,et al.  Bayesian pursuit algorithms , 2010, 2010 18th European Signal Processing Conference.

[12]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[13]  Yongtian Wang,et al.  Robust Photometric Stereo via Low-Rank Matrix Completion and Recovery , 2010, ACCV.

[14]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[16]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Jianjiang Feng,et al.  Smooth Representation Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[19]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[20]  Cédric Herzet,et al.  Sparse representation algorithms based on mean-field approximations , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[22]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[23]  Matthijs Douze,et al.  Large-scale image classification with trace-norm regularization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Narendra Ahuja,et al.  Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-Rank Matrices , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Na Chen,et al.  Error Analysis for Matrix Elastic-Net Regularization Algorithms , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Y. Zhang,et al.  Augmented Lagrangian alternating direction method for matrix separation based on low-rank factorization , 2014, Optim. Methods Softw..

[27]  Shuicheng Yan,et al.  Practical low-rank matrix approximation under robust L1-norm , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  王晓刚,et al.  Background Subtraction via Robust Dictionary Learning , 2011 .

[29]  Nojun Kwak,et al.  Efficient $l_{1}$ -Norm-Based Low-Rank Matrix Approximations for Large-Scale Problems Using Alternating Rectified Gradient Method , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Songhwai Oh,et al.  Elastic-net regularization of singular values for robust subspace learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Shuicheng Yan,et al.  Multi-task low-rank affinity pursuit for image segmentation , 2011, 2011 International Conference on Computer Vision.

[33]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[34]  Alexandre Bernardino,et al.  Unifying Nuclear Norm and Bilinear Factorization Approaches for Low-Rank Matrix Decomposition , 2013, 2013 IEEE International Conference on Computer Vision.

[35]  Aaron Hertzmann,et al.  Learning Non-Rigid 3D Shape from 2D Motion , 2003, NIPS.

[36]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[38]  Charles Soussen,et al.  Joint K-Step Analysis of Orthogonal Matching Pursuit and Orthogonal Least Squares , 2011, IEEE Transactions on Information Theory.

[39]  Anders P. Eriksson,et al.  Robust Weighted Low-Rank Matrix Approximation , 2011 .

[40]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[42]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[43]  Rama Chellappa,et al.  Large-Scale Matrix Factorization with Missing Data under Additional Constraints , 2010, NIPS.

[44]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[45]  Tommi S. Jaakkola,et al.  Weighted Low-Rank Approximations , 2003, ICML.

[46]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[47]  Roberto Tron RenVidal A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007 .

[48]  Takeo Kanade,et al.  Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[49]  Andrew W. Fitzgibbon,et al.  Damped Newton algorithms for matrix factorization with missing data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).