Hierarchical K-means Clustering Using New Support Vector Machines for Multi-class Classification

We propose a binary hierarchical classification structure to address the multi-class classification problem with a new hierarchical design method, k-means SVRM (support vector representation machine) clustering. This greatly improves upon our prior IJCNN hierarchical design. At each node in the hierarchy, we apply the SVRDM (support vector representation and discrimination machine) classifier, which offers generalization and good rejection ability. We also provide new theoretical bases and methods for our choice of the kernel function and new SVRDM parameter selection rules. Classification and rejection test results are presented on new databases of both simulated and real infra-red (IR) data.

[1]  Thomas S. Huang,et al.  One-class SVM for learning in image retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[2]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[3]  Jennifer G. Dy,et al.  A hierarchical method for multi-class support vector machines , 2004, ICML.

[4]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[5]  D. Casasent,et al.  Support vector machines for class representation and discrimination , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[6]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[7]  David Casasent,et al.  Face recognition and verification with pose and illumination variations and imposter rejection , 2005, SPIE Defense + Commercial Sensing.

[8]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[9]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[10]  David G. Stork,et al.  Pattern Classification , 1973 .

[11]  D. Casasent,et al.  Automatic target recognition using new support vector machine , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[12]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[13]  Kishan G. Mehrotra,et al.  Efficient classification for multiclass problems using modular neural networks , 1995, IEEE Trans. Neural Networks.

[14]  Joydeep Ghosh,et al.  Hierarchical Fusion of Multiple Classifiers for Hyperspectral Data Analysis , 2002, Pattern Analysis & Applications.

[15]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[16]  Robert P. W. Duin,et al.  Data domain description using support vectors , 1999, ESANN.