A Nonparametric Model for Multi-Manifold Clustering with Mixture of Gaussians and Graph Consistency

Multi-manifold clustering is among the most fundamental tasks in signal processing and machine learning. Although the existing multi-manifold clustering methods are quite powerful, learning the cluster number automatically from data is still a challenge. In this paper, a novel unsupervised generative clustering approach within the Bayesian nonparametric framework has been proposed. Specifically, our manifold method automatically selects the cluster number with a Dirichlet Process (DP) prior. Then, a DP-based mixture model with constrained Mixture of Gaussians (MoG) is constructed to handle the manifold data. Finally, we integrate our model with the k-nearest neighbor graph to capture the manifold geometric information. An efficient optimization algorithm has also been derived to do the model inference and optimization. Experimental results on synthetic datasets and real-world benchmark datasets exhibit the effectiveness of this new DP-based manifold method.

[1]  Guy Rosman,et al.  The Manhattan Frame Model—Manhattan World Inference in the Space of Surface Normals , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Francesc Moreno-Noguer,et al.  3D Human Pose Tracking Priors using Geodesic Mixture Models , 2017, International Journal of Computer Vision.

[3]  Dinh Q. Phung,et al.  Bayesian Nonparametric Multilevel Clustering with Group-Level Contexts , 2014, ICML.

[4]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[5]  Robert Pless,et al.  Manifold clustering , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Bin Luo,et al.  Permutation Preference Based Alternate Sampling and Clustering for Motion Segmentation , 2018, IEEE Signal Processing Letters.

[7]  Yao Wang,et al.  Low-Rank Matrix Factorization under General Mixture Noise Distributions , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[9]  Aimin Zhou,et al.  Simultaneous Bayesian Clustering and Feature Selection Through Student’s ${t}$ Mixtures Model , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Zoubin Ghahramani,et al.  A nonparametric variable clustering model , 2012, NIPS.

[11]  Erik Blasch,et al.  Simultaneous Trajectory Association and Clustering for Motion Segmentation , 2018, IEEE Signal Processing Letters.

[12]  Oskar Söderkvist,et al.  Computer Vision Classification of Leaves from Swedish Trees , 2001 .

[13]  Wenke Zang,et al.  A Kernel-Based Intuitionistic Fuzzy C-Means Clustering Using a DNA Genetic Algorithm for Magnetic Resonance Image Segmentation , 2017, Entropy.

[14]  Wei-Yun Yau,et al.  Deep Subspace Clustering with Sparsity Prior , 2016, IJCAI.

[15]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[16]  Lei Zhang,et al.  Robust Principal Component Analysis with Complex Noise , 2014, ICML.

[17]  A. Munk,et al.  INTRINSIC SHAPE ANALYSIS: GEODESIC PCA FOR RIEMANNIAN MANIFOLDS MODULO ISOMETRIC LIE GROUP ACTIONS , 2007 .

[18]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[19]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Radford M. Neal Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[21]  Vladimir Pavlovic,et al.  Probabilistic Temporal Subspace Clustering , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Alfred O. Hero,et al.  Phase Transitions in Spectral Community Detection , 2014, IEEE Transactions on Signal Processing.

[23]  Jun Zhu,et al.  DP-space: Bayesian Nonparametric Subspace Clustering with Small-variance Asymptotics , 2015, ICML.

[24]  Søren Hauberg,et al.  Manifold Valued Statistics, Exact Principal Geodesic Analysis and the Effect of Linear Approximations , 2010, ECCV.

[25]  Deepak Aggarwal,et al.  Color Image Segmentation using CIELab Color Space using Ant Colony Optimization , 2011 .

[26]  Jonathan P. How,et al.  Efficient Global Point Cloud Alignment Using Bayesian Nonparametric Mixtures , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Li Guo,et al.  KL Divergence-Based Fuzzy Cluster Ensemble for Image Segmentation , 2018, Entropy.

[28]  Sean Hughes,et al.  Clustering by Fast Search and Find of Density Peaks , 2016 .

[29]  Ralph Roskies,et al.  Fourier Descriptors for Plane Closed Curves , 1972, IEEE Transactions on Computers.

[30]  Deng Cai,et al.  Gaussian Mixture Model with Local Consistency , 2010, AAAI.

[31]  Jiawei Han,et al.  Modeling hidden topics on document manifold , 2008, CIKM '08.

[32]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[33]  Ying Zhang,et al.  Subspace Clustering Under Complex Noise , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Jun Zhu,et al.  BayesianNonparametric Subspace Clustering with Small-variance Asymptotics , 2015 .

[35]  Zhen Yang,et al.  The infinite Student's t-factor mixture analyzer for robust clustering and classification , 2012, Pattern Recognit..

[36]  Mohamed Nadif,et al.  Multi-manifold matrix decomposition for data co-clustering , 2017, Pattern Recognit..

[37]  Hujun Bao,et al.  Laplacian Regularized Gaussian Mixture Model for Data Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[38]  Michael I. Jordan,et al.  Variational inference for Dirichlet process mixtures , 2006 .

[39]  Tong Zhang,et al.  Deep Subspace Clustering Networks , 2017, NIPS.

[40]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Zhijian Wu,et al.  Combined Forecasting of Rainfall Based on Fuzzy Clustering and Cross Entropy , 2017, Entropy.

[42]  A. Munk,et al.  Intrinsic shape analysis: Geodesic principal component analysis for Riemannian manifolds modulo Lie group actions. Discussion paper with rejoinder. , 2010 .

[43]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[44]  Bang Wang,et al.  Occurrence-Based Fingerprint Clustering for Fast Pattern-Matching Location Determination , 2012, IEEE Communications Letters.

[45]  Y. Jiang,et al.  Spectral Clustering on Multiple Manifolds , 2011, IEEE Transactions on Neural Networks.

[46]  Murat Erisoglu,et al.  An Approach for Determining the Number of Clusters in a Model-Based Cluster Analysis , 2017, Entropy.

[47]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Taymaz Rahkar-Farshi,et al.  Image Clustering with Optimization Algorithms and Color Space , 2018, Entropy.