Robust Subspace Clustering by Bi-Sparsity Pursuit: Guarantees and Sequential Algorithm

We consider subspace clustering under sparse noise, for which a non-convex optimization framework based on sparse data representations has been recently developed. This setup is suitable for a large variety of applications with high dimensional data, such as image processing, which is naturally decomposed into a sparse unstructured foreground and a background residing in a union of low-dimensional subspaces. In this framework, we further discuss both performance and implementation of the key optimization problem. We provide an analysis of this optimization problem demonstrating that our approach is capable of recovering linear subspaces as a local optimal solution for sufficiently large data sets and sparse noise vectors. We also propose a sequential algorithmic solution, which is particularly useful for extremely large data sets and online vision applications such as video processing.

[1]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[2]  Dimitri P. Bertsekas,et al.  Incremental proximal methods for large scale convex optimization , 2011, Math. Program..

[3]  A. Butte,et al.  Microarrays for an Integrative Genomics , 2002 .

[4]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[5]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[6]  Emmanuel J. Candès,et al.  A Geometric Analysis of Subspace Clustering with Outliers , 2011, ArXiv.

[7]  Hamid Krim,et al.  BI-sparsity pursuit for robust subspace recovery , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[8]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[9]  Soon Ki Jung,et al.  Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset , 2015, Comput. Sci. Rev..

[10]  René Vidal,et al.  Kernel sparse subspace clustering , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[11]  Soon Ki Jung,et al.  Background–Foreground Modeling Based on Spatiotemporal Sparse Subspace Clustering , 2017, IEEE Transactions on Image Processing.

[12]  Christos Thrampoulidis,et al.  Precise error analysis of the LASSO , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Jeffrey T. Chang,et al.  Basic microarray analysis: grouping and feature reduction. , 2001, Trends in biotechnology.

[14]  Huan Liu,et al.  Subspace clustering for high dimensional data: a review , 2004, SKDD.

[15]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  Brendt Wohlberg,et al.  A Matlab implementation of a fast incremental principal component pursuit algorithm for Video Background Modeling , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[17]  Emmanuel J. Candès,et al.  Robust Subspace Clustering , 2013, ArXiv.

[18]  Dimitrios Gunopulos,et al.  Automatic subspace clustering of high dimensional data for data mining applications , 1998, SIGMOD '98.

[19]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[20]  Helmut Bölcskei,et al.  Robust Subspace Clustering via Thresholding , 2013, IEEE Transactions on Information Theory.

[21]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Hamid Krim,et al.  Robust Subspace Recovery via Bi-Sparsity Pursuit , 2018 .

[23]  Hans-Peter Kriegel,et al.  Clustering high-dimensional data: A survey on subspace clustering, pattern-based clustering, and correlation clustering , 2009, TKDD.

[24]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Martin Kleinsteuber,et al.  pROST: a smoothed $$\ell _p$$ℓp-norm robust online subspace tracking method for background subtraction in video , 2013, Machine Vision and Applications.

[26]  Guillermo Sapiro,et al.  Learning Robust Subspace Clustering , 2013, ArXiv.

[27]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[28]  W. Eric L. Grimson,et al.  Unsupervised Activity Perception in Crowded and Complicated Scenes Using Hierarchical Bayesian Models , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Junbin Gao,et al.  Subspace Clustering for Sequential Data , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Roger L. Berger,et al.  Testing Hypotheses concerning Unions of Linear Subspaces , 1984 .

[31]  René Vidal,et al.  Multiframe Motion Segmentation with Missing Data Using PowerFactorization and GPCA , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[32]  Jianhong Wu,et al.  Subspace clustering for high dimensional categorical data , 2004, SKDD.