Investigation of strategies for the generation of multiclass support vector machines

Support Vector Machines constitute a Machine Learning technique originally designed for the solution of two-class problems. This paper investigates and proposes strategies for the generalization of SVMs to problems with more than two classes. The focus of this work is on strategies that decompose the original multiclass problem into binary subtasks, whose outputs are combined. The proposed strategies aim to investigate the adaptation of the decompositions for each multiclass application considered, using information of the performance obtained in its solution or extracted from its examples. The implemented algorithms were evaluated using benchmark datasets and real applications from the Bioinformatics domain. Among the benefits observed is the obtainment of simpler decompositions, which require less binary classifiers in the multiclass solution.

[1]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[2]  Boonserm Kijsirikul,et al.  Multiclass support vector machines using adaptive directed acyclic graph , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[3]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[4]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[5]  Ravindra K. Ahuja,et al.  Network Flows: Theory, Algorithms, and Applications , 1993 .

[6]  Koby Crammer,et al.  On the Learnability and Design of Output Codes for Multiclass Problems , 2002, Machine Learning.

[7]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[8]  A. J. Feelders,et al.  On the Statistical Comparison of Inductive Learning Methods , 1995, AISTATS.

[9]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[10]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Multiclass SVM Design and Parameter Selection with Genetic Algorithms , 2006, 2006 Ninth Brazilian Symposium on Neural Networks (SBRN'06).

[11]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[12]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[13]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Minimum Spanning Trees in Hierarchical Multiclass Support Vector Machines Generation , 2005, IEA/AIE.

[14]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Evolutionary design of multiclass support vector machines , 2007, J. Intell. Fuzzy Syst..

[15]  Moonis Ali,et al.  Innovations in Applied Artificial Intelligence , 2005 .

[16]  Jason Weston,et al.  Support vector machines for multi-class pattern recognition , 1999, ESANN.

[17]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Protein cellular localization prediction with Support Vector Machines and Decision Trees , 2007, Comput. Biol. Medicine.

[18]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[19]  Doug Fisher,et al.  Learning from Data: Artificial Intelligence and Statistics V , 1996 .

[20]  S. Abe,et al.  Decision-tree-based multiclass support vector machines , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[21]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[22]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Protein Cellular Localization with Multiclass Support Vector Machines and Decision Trees , 2005, BSB.

[23]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[24]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[25]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[26]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[27]  Jirí Benes,et al.  On neural networks , 1990, Kybernetika.

[28]  Jennifer G. Dy,et al.  A hierarchical method for multi-class support vector machines , 2004, ICML.