Learning representative exemplars using one-class Gaussian process regression

Abstract An exemplar is an observation that represents a group of similar observations. Exemplars from data are examined to divide entire heterogeneous data into several homogeneous subgroups, wherein each subgroup is represented by an exemplar. With its inherent sparsity, an exemplar-based learning model provides a parsimonious model to represent or cluster large-scale data. A novel exemplar learning method using one-class Gaussian process (GP) regression is proposed in this study. The proposed method constructs data distribution support from one-class GP regression using automatic relevance determination prior and heterogeneous GP noise. Exemplars that correspond to the basis vectors of the constructed support function are then automatically located during the training process. The proposed method is applied to various data sets to examine its operability, characteristics of data representation, and cluster analysis. The exemplars of some real data generated by the proposed method are also reported.

[1]  Michael E. Tipping.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[2]  Hava T. Siegelmann,et al.  Support Vector Clustering , 2002, J. Mach. Learn. Res..

[3]  Daewon Lee,et al.  An improved cluster labeling method for support vector clustering , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[5]  Peter J. Rousseeuw,et al.  Clustering by means of medoids , 1987 .

[6]  Kyu-Hwan Jung,et al.  Dynamic pattern denoising method using multi-basin system with kernels , 2011, Pattern Recognit..

[7]  Xian-Sheng Hua,et al.  Finding image exemplars using fast sparse affinity propagation , 2008, ACM Multimedia.

[8]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[9]  Chih-Kai Chang,et al.  Feature genes selection of adult ALL microarray data with affinity propagation clustering , 2015, 2015 IEEE International Conference on Consumer Electronics - Taiwan.

[10]  Karen M. Daniels,et al.  Gaussian kernel width exploration and cone cluster labeling for support vector clustering , 2011, Pattern Analysis and Applications.

[11]  Baoguo Yang,et al.  Kernel Based K-Medoids for Clustering Data with Uncertainty , 2010, ADMA.

[12]  Daewon Lee,et al.  Fast support-based clustering method for large-scale problems , 2010, Pattern Recognit..

[13]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Jianhua Yang,et al.  Support vector clustering through proximity graph modelling , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[15]  Hae-Sang Park,et al.  A simple and fast algorithm for K-medoids clustering , 2009, Expert Syst. Appl..

[16]  Fei Wang,et al.  Fast affinity propagation clustering: A multilevel approach , 2012, Pattern Recognit..

[17]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[18]  Daewon Lee,et al.  Dynamic Characterization of Cluster Structures for Robust and Inductive Support Vector Clustering , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[20]  Hildur Ólafsdóttir,et al.  Robust Pseudo-hierarchical Support Vector Clustering , 2007, SCIA.

[21]  Michael E. Tipping,et al.  Fast Marginal Likelihood Maximisation for Sparse Bayesian Models , 2003 .

[22]  Kyu-Hwan Jung,et al.  Probabilistic generative ranking method based on multi-support vector domain description , 2013, Inf. Sci..

[23]  Sheng-De Wang,et al.  Choosing the kernel parameters for support vector machines by the inter-cluster distance in the feature space , 2009, Pattern Recognit..

[24]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[25]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[26]  Kyoungok Kim,et al.  Nonlinear Dynamic Projection for Noise Reduction of Dispersed Manifolds , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Chang-Dong Wang,et al.  Multi-Exemplar Affinity Propagation , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[29]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[30]  Jaewook Lee,et al.  Clustering Based on Gaussian Processes , 2007, Neural Computation.

[31]  R. Krishnapuram,et al.  A fuzzy relative of the k-medoids algorithm with application to web document and snippet clustering , 1999, FUZZ-IEEE'99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No.99CH36315).

[32]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[33]  S. Abe,et al.  Spatially chunking support vector clustering algorithm , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[34]  Michael J. Brusco,et al.  Affinity Propagation and Uncapacitated Facility Location Problems , 2015, Journal of Classification.

[35]  Hongyuan Zha,et al.  Support vector clustering combined with spectral graph partitioning , 2004, ICPR 2004.

[36]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[37]  J. Rogan,et al.  Forest fragmentation in Massachusetts, USA: a town-level assessment using Morphological spatial pattern analysis and affinity propagation , 2016 .

[38]  Kyoungok Kim,et al.  Voronoi Cell-Based Clustering Using a Kernel Support , 2015, IEEE Transactions on Knowledge and Data Engineering.

[39]  Daewon Lee,et al.  Dynamic Dissimilarity Measure for Support-Based Clustering , 2010, IEEE Transactions on Knowledge and Data Engineering.

[40]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[41]  David P. Wipf,et al.  A New View of Automatic Relevance Determination , 2007, NIPS.