A new method for hierarchical clustering combination

In the field of pattern recognition, combining different classifiers into a robust classifier is a common approach for improving classification accuracy. Recently, this trend has also been used to improve clustering performance especially in non-hierarchical clustering approaches. Generally hierarchical clustering is preferred in comparison with the partitional clustering for applications when the exact number of the clusters is not determined or when we are interested in finding the relation between clusters. To the best of our knowledge clustering combination methods proposed so far are based on partitional clustering and hierarchical clustering has been ignored. In this paper, a new method for combining hierarchical clustering is proposed. In this method, in the first step the primary hierarchical clustering dendrograms are converted to matrices. Then these matrices, which describe the dendrograms, are aggregated (using the matrix summation operator) into a final matrix with which the final clustering is formed. The effectiveness of different well known dendrogram descriptors and the one proposed by us for representing the dendrograms are evaluated and compared. The results show that all these descriptor work well and more accurate results (hierarchy of clusters) are obtained using hierarchical combination than combination of partitional clusterings.

[1]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[2]  H. Charles Romesburg,et al.  Cluster analysis for researchers , 1984 .

[3]  Corinna Cortes,et al.  Boosting Decision Trees , 1995, NIPS.

[4]  Hillol Kargupta,et al.  Constructing Simpler Decision Trees from Ensemble Models Using Fourier Analysis , 2002, DMKD.

[5]  Zhi-Hua Zhou,et al.  Ensembling local learners ThroughMultimodal perturbation , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[7]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[8]  Frank Nielsen,et al.  On weighting clustering , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Aristides Gionis,et al.  Clustering aggregation , 2005, 21st International Conference on Data Engineering (ICDE'05).

[10]  Anil K. Jain,et al.  Combining multiple weak clusterings , 2003, Third IEEE International Conference on Data Mining.

[11]  Sandrine Dudoit,et al.  Bagging to Improve the Accuracy of A Clustering Procedure , 2003, Bioinform..

[12]  Hillol Kargupta,et al.  A Fourier spectrum-based approach to represent decision trees for mining data streams in mobile environments , 2004, IEEE Transactions on Knowledge and Data Engineering.

[13]  David G. Stork,et al.  Pattern Classification , 1973 .

[14]  Haimonti Dutta,et al.  Orthogonal decision trees , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[15]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Ana L. N. Fred,et al.  Robust data clustering , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[17]  Michalis Vazirgiannis,et al.  c ○ 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. On Clustering Validation Techniques , 2022 .

[18]  William F. Punch,et al.  Ensembles of partitions via data resampling , 2004, International Conference on Information Technology: Coding and Computing, 2004. Proceedings. ITCC 2004..

[19]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[20]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[21]  Andreas Stafylopatis,et al.  A clustering method based on boosting , 2004, Pattern Recognit. Lett..

[22]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[23]  Ludmila I. Kuncheva,et al.  Moderate diversity for better cluster ensembles , 2006, Inf. Fusion.

[24]  János Podani Simulation of Random Dendrograms and Comparison Tests: Some Comments , 2000, J. Classif..

[25]  Ana L. N. Fred,et al.  Data clustering using evidence accumulation , 2002, Object recognition supported by user interaction for service robots.

[26]  Ana L. N. Fred,et al.  Combining multiple clusterings using evidence accumulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Anil K. Jain,et al.  A Mixture Model for Clustering Ensembles , 2004, SDM.

[28]  Ana L. N. Fred,et al.  Finding Consistent Clusters in Data Partitions , 2001, Multiple Classifier Systems.

[29]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[30]  Ludmila I. Kuncheva,et al.  Experimental Comparison of Cluster Ensemble Methods , 2006, 2006 9th International Conference on Information Fusion.

[31]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[32]  George Karypis,et al.  Evaluation of hierarchical clustering algorithms for document datasets , 2002, CIKM '02.

[33]  Mohamed S. Kamel,et al.  Finding Natural Clusters Using Multi-clusterer Combiner Based on Shared Nearest Neighbors , 2003, Multiple Classifier Systems.

[34]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[35]  Wei Tang,et al.  Clusterer ensemble , 2006, Knowl. Based Syst..

[36]  R. Schapire The Strength of Weak Learnability , 1990, Machine Learning.

[37]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[38]  Jill P. Mesirov,et al.  Consensus Clustering: A Resampling-Based Method for Class Discovery and Visualization of Gene Expression Microarray Data , 2003, Machine Learning.

[39]  Isabelle Guyon,et al.  A Stability Based Method for Discovering Structure in Clustered Data , 2001, Pacific Symposium on Biocomputing.

[40]  Joydeep Ghosh,et al.  Cluster Ensembles A Knowledge Reuse Framework for Combining Partitionings , 2002, AAAI/IAAI.

[41]  Carla E. Brodley,et al.  Solving cluster ensemble problems by bipartite graph partitioning , 2004, ICML.

[42]  Michael Georgiopoulos,et al.  Boosted ARTMAP: Modifications to fuzzy ARTMAP motivated by boosting theory , 2006, Neural Networks.

[43]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[44]  Kurt Hornik,et al.  An Ensemble Method for Clustering , 2003 .

[45]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[46]  Joachim M. Buhmann,et al.  Bagging for Path-Based Clustering , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[48]  Yang Yu,et al.  Ensembling local learners ThroughMultimodal perturbation , 2005, IEEE Trans. Syst. Man Cybern. Part B.

[49]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[50]  Anil K. Jain,et al.  Clustering ensembles: models of consensus and weak partitions , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[52]  Anil K. Jain,et al.  Adaptive clustering ensembles , 2004, ICPR 2004.

[53]  François-Joseph Lapointe,et al.  A STATISTICAL FRAMEWORK TO TEST THE CONSENSUS AMONG ADDITIVE TREES (CLADOGRAMS) , 1992 .