Mathematical programming based heuristics for improving LP-generated classifiers for the multiclass supervised classification problem

Mathematical programming is used as a nonparametric approach to supervised classification. However, mathematical programming formulations that minimize the number of misclassifications on the design dataset suffer from computational difficulties. We present mathematical programming based heuristics for finding classifiers with a small number of misclassifications on the design dataset. The basic idea is to improve an LPgenerated classifier with respect to the number of misclassifications on the design dataset. The heuristics are evaluated computationally on both simulated and real world datasets.

[1]  Johan A. K. Suykens,et al.  Benchmarking Least Squares Support Vector Machine Classifiers , 2004, Machine Learning.

[2]  O. Mangasarian,et al.  Multicategory discrimination via linear programming , 1994 .

[3]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[4]  Robert P. W. Duin,et al.  Using two-class classifiers for multiclass classification , 2002, Object recognition supported by user interaction for service robots.

[5]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[6]  Patrice Marcotte,et al.  A new implicit enumeration scheme for the discriminant analysis problem , 1995, Comput. Oper. Res..

[7]  Robert C. Soltysik,et al.  The WARMACK-GONZALEZ algorithm for linear two-category multivariable optimal discriminant analysis , 1994, Comput. Oper. Res..

[8]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[9]  Paul A. Rubin,et al.  Solving mixed integer classification problems by decomposition , 1997, Ann. Oper. Res..

[10]  V. Srinivasan,et al.  Multigroup Discriminant Analysis Using Linear Programming , 1997, Oper. Res..

[11]  Paul A. Rubin,et al.  Heuristic solution procedures for a mixed‐integer programming discriminant model , 1990 .

[12]  William J. Banks,et al.  An Efficient Optimal Solution Algorithm for the Classification Problem , 1991 .

[13]  Wei-Yin Loh,et al.  A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms , 2000, Machine Learning.

[14]  Gary J. Koehler,et al.  Minimizing Misclassifications in Linear Discriminant Analysis , 1990 .

[15]  David J. Hand,et al.  Construction and Assessment of Classification Rules , 1997 .

[16]  John M. Liittschwager,et al.  Integer Programming Solution of a Classification Problem , 1978 .

[17]  W. Gehrlein.  General mathematical programming formulations for the statistical classification problem , 1986 .

[18]  Antonie Stam,et al.  A comparison of a robust mixed-integer approach to existing methods for establishing classification rules for the discriminant problem , 1990 .

[19]  S. M. Bajgier,et al.  AN EXPERIMENTAL COMPARISON OF STATISTICAL AND LINEAR PROGRAMMING APPROACHES TO THE DISCRIMINANT PROBLEM , 1982 .

[20]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[21]  S. Selcuk Erenguc,et al.  Survey of mathematical programming models and experimental results for linear discriminant analysis , 1990 .

[22]  Kristin P. Bennett,et al.  A Parametric Optimization Method for Machine Learning , 1997, INFORMS J. Comput..

[23]  F. Glover,et al.  Simple but powerful goal programming models for discriminant problems , 1981 .

[24]  John W. Chinneck,et al.  Fast Heuristics for the Maximum Feasible Subsystem Problem , 2001, INFORMS J. Comput..

[25]  Gary J. Koehler,et al.  Linear Discriminant Functions Determined by Genetic Search , 1991, INFORMS J. Comput..

[26]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .