A Hierarchical Cluster Validity Based Visual Tree Learning for Hierarchical Classification

For hierarchical learning, one open issue is how to build a reasonable hierarchical structure which characterize the inter-relation between categories. An effective approach is to utilize hierarchical clustering to build a visual tree structure, however, the critical issue of this approach is how to determine the number of clusters in hierarchical clustering. In this paper, a hierarchical cluster validity index (HCVI) is developed for supporting visual tree learning. Before clustering of each level begins, we will measure the impact of different numbers of clusters on visual tree building and select the most suitable number of clusters. The proposed HCVI will control the structure of visual tree neither too flat nor too deep. Based on this visual tree, a hierarchical classifier can be trained for achieving more discriminative capability. Our experimental results have demonstrated that the proposed hierarchical cluster validity index (HCVI) can guide the building of a more reasonable visual tree structure, so that the hierarchical classifier can achieve better results on classification accuracy.

[1]  Jason Weston,et al.  Label Embedding Trees for Large Multi-Class Tasks , 2010, NIPS.

[2]  Nan Liu,et al.  Knowledge Acquisition and Representation Using Fuzzy Evidential Reasoning and Dynamic Adaptive Fuzzy Petri Nets , 2013, IEEE Transactions on Cybernetics.

[3]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[4]  Alexander C. Berg,et al.  Fast and Balanced: Efficient Label Tree Learning for Large Scale Object Recognition , 2011, NIPS.

[5]  C. Mallows,et al.  A Method for Comparing Two Hierarchical Clusterings , 1983 .

[6]  Michael K. Ng,et al.  ML-FOREST: A Multi-Label Tree Ensemble Method for Multi-Label Classification , 2016, IEEE Transactions on Knowledge and Data Engineering.

[7]  Yang Wei,et al.  Low-Rank Latent Pattern Approximation With Applications to Robust Image Classification , 2017, IEEE Transactions on Image Processing.

[8]  Ohad Shamir,et al.  Probabilistic Label Trees for Efficient Large Scale Image Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Fei-Fei Li,et al.  Building and using a semantivisual image hierarchy , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Li Lin,et al.  Joint Hierarchical Category Structure Learning and Large-Scale Image Classification , 2017, IEEE Transactions on Image Processing.

[11]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[12]  T. Caliński,et al.  A dendrite method for cluster analysis , 1974 .

[13]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[14]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Hui Xiong,et al.  Understanding and Enhancement of Internal Clustering Validation Measures , 2013, IEEE Transactions on Cybernetics.

[16]  Huy Phan,et al.  Label Tree Embeddings for Acoustic Scene Classification , 2016, ACM Multimedia.

[17]  Ioannis A. Kakadiaris,et al.  Hierarchical Multi-label Classification using Fully Associative Ensemble Learning , 2017, Pattern Recognit..

[18]  Hui Xiong,et al.  Understanding of Internal Clustering Validation Measures , 2010, 2010 IEEE International Conference on Data Mining.

[19]  Jianping Fan,et al.  Hierarchical learning of multi-task sparse metrics for large-scale image classification , 2017, Pattern Recognit..

[20]  Pietro Perona,et al.  Learning and using taxonomies for fast visual categorization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Robert Tibshirani,et al.  Estimating the number of clusters in a data set via the gap statistic , 2000 .

[22]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[23]  Camille Roth,et al.  Natural Scales in Geographical Patterns , 2017, Scientific Reports.

[24]  Jianping Fan,et al.  Hierarchical Learning of Tree Classifiers for Large-Scale Plant Species Identification , 2015, IEEE Transactions on Image Processing.

[25]  Malika Charrad,et al.  NbClust package: finding the relevant number of clusters in a dataset , 2012 .

[26]  Yang Lei,et al.  Ground truth bias in external cluster validity indices , 2016, Pattern Recognit..

[27]  Jianping Fan,et al.  Integrating Concept Ontology and Multitask Learning to Achieve More Effective Classifier Training for Multilevel Image Annotation , 2008, IEEE Transactions on Image Processing.

[28]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[29]  Majid Sarrafzadeh,et al.  A Hierarchical Classification and Segmentation Scheme for Processing Sensor Data , 2017, IEEE Journal of Biomedical and Health Informatics.

[30]  Jianping Fan,et al.  Jointly Learning Visually Correlated Dictionaries for Large-Scale Visual Recognition Applications , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  P. Jaccard,et al.  Etude comparative de la distribution florale dans une portion des Alpes et des Jura , 1901 .

[32]  Qi Tian,et al.  LEGO-MM: LEarning Structured Model by Probabilistic loGic Ontology Tree for MultiMedia , 2017, IEEE Transactions on Image Processing.

[33]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[34]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[35]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[36]  Liuqing Peng,et al.  CVAP: Validation for Cluster Analyses , 2009, Data Sci. J..

[37]  Ya-Feng Liu,et al.  LLE Score: A New Filter-Based Unsupervised Feature Selection Method Based on Nonlinear Manifold Embedding and Its Application to Image Recognition , 2017, IEEE Transactions on Image Processing.

[38]  Maria A. Zuluaga,et al.  Detecting Clinically Meaningful Shape Clusters in Medical Image Data: Metrics Analysis for Hierarchical Clustering Applied to Healthy and Pathological Aortic Arches , 2017, IEEE Transactions on Biomedical Engineering.

[39]  Cordelia Schmid,et al.  Constructing Category Hierarchies for Visual Recognition , 2008, ECCV.

[40]  Jianping Fan,et al.  Deep Mixture of Diverse Experts for Large-Scale Visual Recognition , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  J. Dunn Well-Separated Clusters and Optimal Fuzzy Partitions , 1974 .