Bayesian Low-Rank and Sparse Nonlinear Representation for Manifold Clustering

Linear representation usually used by the optimization model about low-rankness and sparsity limits their applications to some extent. In this paper, we propose Bayesian low-rank and sparse nonlinear representation (BLSN) model exploiting nonlinear representation. Different from the optimization model, BLSN can be solved by traditional algorithm in Bayesian statistics easily without knowing the explicit mapping by kernel trick. Moreover, it can learn the parameters adaptively to choose the low-rank and sparse properties and also provides a way to enforce more properties on one quantity in a Bayesian model. Based on the observation that the data points drawn from a union of manifolds may gain more meaningful linear structure after a nonlinear mapping, we apply BLSN for manifold clustering. It can handle different problems by constructing various kernels. With respect to the case of linear manifold, known as subspace segmentation, we propose a kernel by the Veronese mapping. In addition, we also design the kernel matrices for the case of nonlinear manifold. Experimental results confirm the effectiveness and the potential of our model for manifold clustering.

[1]  J. Griffin,et al.  Inference with normal-gamma prior distributions in regression problems , 2010 .

[2]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[3]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[4]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Roberto Tron RenVidal A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007 .

[6]  Jaeyong Lee,et al.  GENERALIZED DOUBLE PARETO SHRINKAGE. , 2011, Statistica Sinica.

[7]  Jian Yu,et al.  Saliency Detection by Multitask Sparsity Pursuit , 2012, IEEE Transactions on Image Processing.

[8]  Yong Wang,et al.  Multi-manifold Clustering , 2010, PRICAI.

[9]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  René Vidal,et al.  Kernel sparse subspace clustering , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[11]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Jie Zhang,et al.  Structure-Constrained Low-Rank Representation , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[14]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[15]  Robert Pless,et al.  Manifold clustering , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[16]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[17]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[18]  James G. Scott,et al.  The horseshoe estimator for sparse signals , 2010 .

[19]  J KriegmanDavid,et al.  Acquiring Linear Subspaces for Face Recognition under Variable Lighting , 2005 .

[20]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Jim Jing-Yan Wang,et al.  Feature selection and multi-kernel learning for sparse representation on a manifold , 2014, Neural Networks.

[22]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[23]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[24]  Aggelos K. Katsaggelos,et al.  Sparse Bayesian Methods for Low-Rank Matrix Estimation , 2011, IEEE Transactions on Signal Processing.

[25]  J. Leslie The Inverse Gaussian Distribution: Theory, Methodology, and Applications , 1990 .

[26]  Shuicheng Yan,et al.  Correlation Adaptive Subspace Segmentation by Trace Lasso , 2013, 2013 IEEE International Conference on Computer Vision.

[27]  Lawrence Carin,et al.  Bayesian Robust Principal Component Analysis , 2011, IEEE Transactions on Image Processing.

[28]  Chris Hans Bayesian lasso regression , 2009 .

[29]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[30]  Shuicheng Yan,et al.  Robust Subspace Segmentation with Block-Diagonal Prior , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Allen Y. Yang,et al.  Unsupervised segmentation of natural images via lossy data compression , 2008, Comput. Vis. Image Underst..

[33]  Michael I. Jordan,et al.  Mixtures of Probabilistic Principal Component Analyzers , 2001 .

[34]  Xian-Da Zhang,et al.  Matrix Analysis and Applications , 2017 .

[35]  Michael E. Tipping Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[36]  T. Willmore Algebraic Geometry , 1973, Nature.

[37]  Guangliang Chen,et al.  Kernel Spectral Curvature Clustering (KSCC) , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[38]  René Vidal,et al.  Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Narendra Ahuja,et al.  Low-Rank Sparse Learning for Robust Visual Tracking , 2012, ECCV.

[40]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[41]  Matti Pietikäinen,et al.  Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2000, ECCV.

[42]  R. Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications. , 2013, IEEE transactions on pattern analysis and machine intelligence.

[43]  Zhixun Su,et al.  Fixed-rank representation for unsupervised visual learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Emmanuel J. Candès,et al.  A Geometric Analysis of Subspace Clustering with Outliers , 2011, ArXiv.

[46]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[47]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[48]  Jianjiang Feng,et al.  Smooth Representation Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[50]  Y. Jiang,et al.  Spectral Clustering on Multiple Manifolds , 2011, IEEE Transactions on Neural Networks.

[51]  René Vidal,et al.  Clustering disjoint subspaces via sparse representation , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[52]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[53]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[54]  Guangliang Chen,et al.  Spectral Curvature Clustering (SCC) , 2009, International Journal of Computer Vision.

[55]  Mário A. T. Figueiredo Adaptive Sparseness for Supervised Learning , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Shuicheng Yan,et al.  Multi-task low-rank affinity pursuit for image segmentation , 2011, 2011 International Conference on Computer Vision.

[57]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.