Robust recovery of multiple subspaces by geometric l_p minimization

We assume i.i.d. data sampled from a mixture distribution with K components along fixed d-dimensional linear subspaces and an additional outlier component. For p>0, we study the simultaneous recovery of the K fixed subspaces by minimizing the l_p-averaged distances of the sampled data points from any K subspaces. Under some conditions, we show that if $0 1 and p>1, then the underlying subspaces cannot be recovered or even nearly recovered by l_p minimization. The results of this paper partially explain the successes and failures of the basic approach of l_p energy minimization for modeling data by multiple subspaces.

[1]  T. W. Anderson.  An Introduction to Multivariate Statistical Analysis , 1959 .

[2]  D. Pollard.  Strong Consistency of $K$-Means Clustering , 1981 .

[3]  D. Pollard.  A Central Limit Theorem for $k$-Means Clustering , 1982 .

[4]  C. O'Connor.  An introduction to multivariate statistical analysis: 2nd edn. by T. W. Anderson. 675 pp. Wiley, New York (1984) , 1987 .

[5]  Kenneth Falconer,et al.  GEOMETRY OF SETS AND MEASURES IN EUCLIDEAN SPACES FRACTALS AND RECTIFIABILITY , 1996 .

[6]  Michael E. Tipping,et al.  Mixtures of Principal Component Analysers , 1997 .

[7]  P. Torr.  Geometric motion segmentation and model selection , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[8]  Paul S. Bradley,et al.  k-Plane Clustering , 2000, J. Glob. Optim..

[9]  P. Tseng.  Nearest q-Flat to m Points , 2000 .

[10]  Kenichi Kanatani,et al.  Motion segmentation by subspace separation and model selection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[11]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[12]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[13]  Nello Cristianini,et al.  On the eigenspectrum of the gram matrix and the generalization error of kernel-PCA , 2005, IEEE Transactions on Information Theory.

[14]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[16]  Akram Aldroubi,et al.  Optimal Non-Linear Models for Sparsity and Sampling , 2007, 0707.2008.

[17]  John Wright,et al.  Segmentation of multivariate mixed data via lossy coding and compression , 2007, Electronic Imaging.

[18]  John Wright,et al.  Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Guangliang Chen,et al.  Spectral Curvature Clustering (SCC) , 2009, International Journal of Computer Vision.

[20]  Allen Y. Yang,et al.  Estimation of Subspace Arrangements with Applications in Modeling and Segmenting Mixed Data , 2008, SIAM Rev..

[21]  Gilad Lerman,et al.  Median K-Flats for hybrid linear modeling with many outliers , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[22]  Guangliang Chen,et al.  Foundations of a Multi-way Spectral Clustering Framework for Hybrid Linear Modeling , 2008, Found. Comput. Math..

[23]  Gilad Lerman,et al.  $${l_p}$$lp-Recovery of the Most Significant Subspace Among Multiple Subspaces with Outliers , 2010, ArXiv.

[24]  Sayan Mukherjee,et al.  Towards Stratification Learning through Homology Inference , 2010, 1008.3572.

[25]  Gilad Lerman,et al.  Randomized hybrid linear modeling by local best-fit flats , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Guangliang Chen,et al.  Spectral clustering based on local linear approximations , 2010, 1001.1323.

[27]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[28]  T. Tao.  Topics in Random Matrix Theory , 2012 .

[29]  Gilad Lerman,et al.  Hybrid Linear Modeling via Local Best-Fit Flats , 2010, International Journal of Computer Vision.