A multi-scale non-parametric and parametric hybrid multi-category recognition algorithm with probabilistic outputs

The synthesis of an effective multi-category nonlinear classifier with the capability to output calibrated posterior probabilities to enable post-processing is of great significance in practical recognition situations because the posterior probability reflects the assessment uncertainty. However, the estimation of posterior probability for multi-category classifiers is an unwieldy problem in the realm of pattern recognition, which usually is more intractable than that in dichotomic cases. In this paper, with the aid of binary tree representation for nested structures, a new polychotomous classification and posterior probability estimation scheme is developed on the strength of Bayesian decision theory. In particular, by capitalising on the intrinsic conexus between hierarchical structure and multi-scale analysis, the polychotomous multi-scale Bayesian kernel Fisher discriminant (KFD) is implemented for building the classifier at different scales for different levels. Finally, the performance of the proposed classification and posterior probability estimation algorithm is validated by designing a multi-category Bayesian KFD classifier for a benchmark satellite images dataset.

[1]  Tony Belpaeme,et al.  Factors influencing the origins of colour categories , 2002 .

[2]  B. Fei,et al.  Binary tree of SVM: a new fast multiclass training and classification algorithm , 2006, IEEE Transactions on Neural Networks.

[3]  Lorenzo Bruzzone,et al.  A multiple-cascade-classifier system for a robust and partially unsupervised updating of land-cover maps , 2002, IEEE Trans. Geosci. Remote. Sens..

[4]  E. Lehmann.  Elements of large-sample theory , 1998 .

[5]  Haoda Fu,et al.  A Multi-scale Nonparametric / Parametric Hybrid Recognition Strategy with Multi-category Posterior Probability Estimation , 2013 .

[6]  Mohamed Cheriet,et al.  Estimating accurate multi-class probabilities with support vector machines , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[7]  Sami F. Masri,et al.  A hybrid parametric/nonparametric approach for the identification of nonlinear systems , 1994 .

[8]  Bernhard Schölkopf,et al.  Kernel Fisher Discriminant , 2001 .

[9]  J. Woods,et al.  Probability and Random Processes with Applications to Signal Processing , 2001 .

[10]  Soo-Young Lee,et al.  Support Vector Machines with Binary Tree Architecture for Multi-Class Classification , 2004 .

[11]  S. Feyo de Azevedo,et al.  Bioprocess hybrid parametric/nonparametric modelling based on the concept of mixture of experts , 2008 .

[12]  Stephen A. Billings,et al.  Generalized multiscale radial basis function networks , 2007, Neural Networks.

[13]  Gangbing Song,et al.  Polychotomous kernel Fisher discriminant via top-down induction of binary tree , 2010, Comput. Math. Appl..

[14]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[15]  Peter V. Gehler,et al.  Kernel learning approaches for image classification , 2009 .

[16]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[17]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[18]  Heikki Mannila,et al.  Distance measures for point sets and their computation , 1997, Acta Informatica.

[19]  Jason Weston,et al.  Support vector machines for multi-class pattern recognition , 1999, ESANN.

[20]  S. A. Billings,et al.  Forecasting the geomagnetic activity of the Dst index using multiscale radial basis function networks , 2007 .

[21]  Jiaxin Wang,et al.  Non-flat function estimation with a multi-scale support vector regression , 2006, Neurocomputing.

[22]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[23]  Miguel Figueroa,et al.  Competitive learning with floating-gate circuits , 2002, IEEE Trans. Neural Networks.

[24]  Cheng Wang,et al.  Combining Support Vector Machines With a Pairwise Decision Tree , 2008, IEEE Geoscience and Remote Sensing Letters.

[25]  Jian Yang,et al.  Essence of kernel Fisher discriminant: KPCA plus LDA , 2004, Pattern Recognit..

[26]  David Casasent,et al.  A hierarchical classifier using new support vector machines for automatic target recognition , 2005, Neural Networks.

[27]  Guangtao Zhai,et al.  Image Reconstruction From Random Samples With Multiscale Hybrid Parametric and Nonparametric Modeling , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Daphna Weinshall,et al.  Classification with Nonmetric Distances: Image Retrieval and Class Representation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[30]  P. Bartlett,et al.  Probabilities for SV Machines , 2000 .

[31]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[32]  Amnon Shashua,et al.  On the Relationship Between the Support Vector Machine for Classification and Sparsified Fisher's Linear Discriminant , 1999, Neural Processing Letters.

[33]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[34]  G. Wahba,et al.  Multicategory Support Vector Machines , Theory , and Application to the Classification of Microarray Data and Satellite Radiance Data , 2004 .

[35]  Hannu Oja,et al.  Classification Based on Hybridization of Parametric and Nonparametric Classifiers , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Bianca Zadrozny,et al.  Transforming classifier scores into accurate multiclass probability estimates , 2002, KDD.

[37]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[38]  Sebastian Mika,et al.  Kernel Fisher Discriminants , 2003 .

[39]  Zhao Lu,et al.  DESIGN OF DECISION TREE VIA KERNELIZED HIERARCHICAL CLUSTERING FOR MULTICLASS SUPPORT VECTOR MACHINES , 2007, Cybern. Syst..

[40]  Hsuan-Tien Lin,et al.  A note on Platt’s probabilistic outputs for support vector machines , 2007, Machine Learning.