Sequential Automatic Search of a Subset of Classifiers in Multiclass Learning

A method called Sequential Automatic Search of a Subset of Classifiers is hereby introduced to deal with classification problems requiring decisions among a wide set of competing classes. It utilizes classifiers in a sequential way by restricting the number of competing classes while maintaining the presence of the true (class) outcome in the candidate set of classes. Some features of the method are discussed, namely: a cross-validation-based criteria to select the best classifier in each iteration of the algorithm, the resulting classification model and the possibility of choosing between an heuristic or probabilistic criteria to predict test set observations. Furthermore, the possibility to cast the whole method in the framework of unsupervised learning is also investigated. Advantages of the method are illustrated analyzing data from a letter recognition experiment.

[1]  H. Sebastian Seung,et al.  Unsupervised Learning by Convex and Conic Coding , 1996, NIPS.

[2]  Younès Bennani,et al.  Dendogram based SVM for multi-class classification , 2006, 28th International Conference on Information Technology Interfaces, 2006..

[3]  Terence C. Fogarty,et al.  Technical Note: First Nearest Neighbor Classification on Frey and Slate's Letter Recognition Problem , 1992, Machine Learning.

[4]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[5]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[6]  Florin Cutzu,et al.  Polychotomous Classification with Pairwise Classifiers: A New Voting Principle , 2003, Multiple Classifier Systems.

[7]  Alexander J. Smola,et al.  Adaptive Margin Support Vector Machines , 2000 .

[8]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[9]  Alexander J. Smola,et al.  Advances in Large Margin Classifiers , 2000 .

[10]  Philipp Koehn,et al.  Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing , 2001, EMNLP 2001.

[11]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[12]  David G. Stork,et al.  Pattern Classification , 1973 .

[13]  Dan Roth,et al.  A Sequential Model for Multi-Class Classification , 2001, EMNLP.

[14]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[15]  Dirk Van den Poel,et al.  Constrained optimization of data-mining problems to improve model performance: A direct-marketing application , 2005, Expert Syst. Appl..

[16]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[17]  David J. Slate,et al.  Letter Recognition Using Holland-Style Adaptive Classifiers , 1991, Machine Learning.