Cluster Ensembles Based on Vector Space Embeddings of Graphs

Cluster ensembles provide us with a versatile alternative to individual clustering algorithms. In structural pattern recognition, however, cluster ensembles have been rarely studied. In the present paper a general methodology for creating structural cluster ensembles is proposed. Our representation formalism is based on graphs and includes strings and trees as special cases. The basic idea of our approach is to view the dissimilarities of an input graph g to a number of prototype graphs as a vectorial description of g . Randomized prototype selection offers a convenient possibility to generate m different vector sets out of the same graph set. Applying any available clustering algorithm to these vector sets results in a cluster ensemble with m clusterings which can then be combined with an appropriate consensus function. In several experiments conducted on different graph sets, the cluster ensemble shows superior performance over two single clustering procedures.

[1]  Kaspar Riesen,et al.  Approximate graph edit distance computation by means of bipartite graph matching , 2009, Image Vis. Comput..

[2]  Sandrine Dudoit,et al.  Bagging to Improve the Accuracy of A Clustering Procedure , 2003, Bioinform..

[3]  Anil K. Jain,et al.  Large-scale evaluation of multimodal biometric authentication using state-of-the-art systems , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Horst Bunke,et al.  A Graph-Theoretic Approach to Enterprise Network Dynamics (Progress in Computer Science and Applied Logic (PCS)) , 2006 .

[5]  Anil K. Jain,et al.  Decision-level fusion in fingerprint verification , 2001, Pattern Recognit..

[6]  Ludmila I. Kuncheva,et al.  Evaluation of Stability of k-Means Cluster Ensembles with Respect to Random Initialization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Chaohong Wu,et al.  Iterative Methods for Searching Optimal Classifier Combination Function , 2007, 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems.

[8]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[9]  Venu Govindaraju,et al.  Optimal Classifier Combination Rules for Verification and Identification Systems , 2007, MCS.

[10]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[11]  Kaspar Riesen,et al.  IAM Graph Database Repository for Graph Based Pattern Recognition and Machine Learning , 2008, SSPR/SPR.

[12]  Edwin R. Hancock,et al.  Structural, Syntactic, and Statistical Pattern Recognition, Joint IAPR International Workshop, SSPR&SPR 2010, Cesme, Izmir, Turkey, August 18-20, 2010. Proceedings , 2010, SSPR/SPR.

[13]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[14]  Ana L. N. Fred,et al.  Combining multiple clusterings using evidence accumulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Sung Bum Pan,et al.  Fusion for Multimodal Biometric Identification , 2005, AVBPA.

[16]  Thomas Gärtner,et al.  Kernels for structured data , 2008, Series in Machine Perception and Artificial Intelligence.

[17]  Tatsuya Akutsu,et al.  Graph Kernels for Molecular Structure-Activity Relationship Analysis with Support Vector Machines , 2005, J. Chem. Inf. Model..

[18]  Venu Govindaraju,et al.  Utilizing Independence of Multimodal Biometric Matchers , 2006, MRCS.

[19]  Alexander H. Waibel,et al.  A novel objective function for improved phoneme recognition using time delay neural networks , 1990, International 1989 Joint Conference on Neural Networks.

[20]  Abraham Kandel,et al.  Graph-Theoretic Techniques for Web Content Mining , 2005, Series in Machine Perception and Artificial Intelligence.

[21]  Gyeonghwan Kim,et al.  A Lexicon Driven Approach to Handwritten Word Recognition for Real-Time Applications , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Venu Govindaraju,et al.  Use of Identification Trial Statistics for the Combination of Biometric Matchers , 2008, IEEE Transactions on Information Forensics and Security.

[23]  Kaspar Riesen,et al.  Graph Classification Based on Vector Space Embedding , 2009, Int. J. Pattern Recognit. Artif. Intell..

[24]  Kaspar Riesen,et al.  Classifier Ensembles for Vector Space Embedding of Graphs , 2007, MCS.

[25]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[26]  Horst Bunke,et al.  Inexact graph matching for structural pattern recognition , 1983, Pattern Recognit. Lett..

[27]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[28]  J. Dunn Well-Separated Clusters and Optimal Fuzzy Partitions , 1974 .

[29]  Biing-Hwang Juang,et al.  Discriminative learning for minimum error classification [pattern recognition] , 1992, IEEE Trans. Signal Process..

[30]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[31]  W. Wallis,et al.  A Graph-Theoretic Approach to Enterprise Network Dynamics , 2006 .

[32]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[33]  L. Hubert,et al.  Quadratic assignment as a general data analysis strategy. , 1976 .

[34]  Vladimir Nedeljkovic A novel multilayer neural networks training algorithm that minimizes the probability of classification error , 1993, IEEE Trans. Neural Networks.

[35]  T. N. Bhat,et al.  The Protein Data Bank , 2000, Nucleic Acids Res..

[36]  Ching Y. Suen,et al.  A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Mohamed S. Kamel,et al.  Finding Natural Clusters Using Multi-clusterer Combiner Based on Shared Nearest Neighbors , 2003, Multiple Classifier Systems.

[38]  Kaspar Riesen,et al.  Kernel k-Means Clustering Applied to Vector Space Embeddings of Graphs , 2008, ANNPR.

[39]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[40]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[41]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Horst Bunke,et al.  Transforming Strings to Vector Spaces Using Prototype Selection , 2006, SSPR/SPR.

[43]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[44]  Naonori Ueda,et al.  Optimal Linear Combination of Neural Networks for Improving Classification Performance , 2000, IEEE Trans. Pattern Anal. Mach. Intell..