A tabu search with an oscillation strategy for the discriminant analysis problem

This article proposes a tabu search approach to solve a mathematical programming formulation of the linear classification problem, which consists of determining an hyperplane that separates two groups of points as well as possible in @?^m. The tabu search approach proposed is based on a non-standard formulation using linear system infeasibility. The search space is the set of bases defined on the matrix that describes the linear system. The moves are performed by pivoting on a specified row and column. On real machine learning databases, our approach compares favorably with implementations based on parametric programming and irreducible infeasible constraint sets. Additional computational results for randomly generated instances confirm that our method provides a suitable alternative to the mixed integer programming formulation that is solved by a commercial code when the number of attributes m increases.

[1]  John J. Glen,et al.  An iterative mixed integer programming method for classification accuracy maximizing discriminant analysis , 2003, Comput. Oper. Res..

[2]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[3]  John M. Liittschwager,et al.  Integer Programming Solution of a Classification Problem , 1978 .

[4]  M. L. Tricot,et al.  Formules de réactualisation pour une famille d'indices de proximité inter-classe en classification hiérarchique , 1989 .

[5]  Gary J. Koehler,et al.  Linear Discriminant Functions Determined by Genetic Search , 1991, INFORMS J. Comput..

[6]  Kim Fung Lam,et al.  An experimental comparison of some recently developed linear programming approaches to the discriminant problem , 1997, Comput. Oper. Res..

[7]  Fred Glover,et al.  A NEW CLASS OF MODELS FOR THE DISCRIMINANT PROBLEM , 1988 .

[8]  Leon Bobrowski.  Linear discrimination with symmetrical models , 1986, Pattern Recognit..

[9]  Koen Bertels,et al.  Qualitative company performance evaluation: Linear discriminant analysis and neural network models , 1999, Eur. J. Oper. Res..

[10]  Raktim Pal,et al.  Predicting the survival or failure of click-and-mortar corporations: A knowledge discovery approach , 2006, Eur. J. Oper. Res..

[11]  Antonie Stam,et al.  Second order mathematical programming formulations for discriminant analysis , 1994 .

[12]  James P. Ignizio,et al.  Discriminant analysis via mathematical programming: Certain problems and their causes , 1989, Comput. Oper. Res..

[13]  Kristin P. Bennett,et al.  A Parametric Optimization Method for Machine Learning , 1997, INFORMS J. Comput..

[14]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[15]  Satosi Watanabe,et al.  Methodologies of Pattern Recognition , 1969 .

[16]  Kim Fung Lam,et al.  Combining discriminant methods in solving classification problems in two-group discriminant analysis , 2002, Eur. J. Oper. Res..

[17]  Patrice Marcotte,et al.  A new implicit enumeration scheme for the discriminant analysis problem , 1995, Comput. Oper. Res..

[18]  A. Goudie,et al.  Forecasting Corporate Failure: The Use of Discriminant Analysis Within a Disaggregated Model of the Corporate Sector , 1987 .

[19]  Prakash L. Abad,et al.  On the performance of linear programming heuristics applied on a quadratic transformation in the classification problem , 1994 .

[20]  Olvi L. Mangasarian,et al.  Mathematical Programming in Neural Networks , 1993, INFORMS J. Comput..

[21]  John M. Wilson,et al.  Integer programming formulations of statistical classification problems , 1996 .

[22]  Cliff T. Ragsdale,et al.  Mathematical Programming Formulations for the Discriminant Problem: An Old Dog Does New Tricks* , 1991 .

[23]  William Beranek,et al.  CREDIT‐SCORING MODELS AND THE CUT‐OFF POINT—A SIMPLIFICATION , 1976 .

[24]  Shingo Tomita,et al.  An optimal orthonormal system for discriminant analysis , 1985, Pattern Recognit..

[25]  William Nick Street,et al.  Breast Cancer Diagnosis and Prognosis Via Linear Programming , 1995, Oper. Res..

[26]  William J. Banks,et al.  An Efficient Optimal Solution Algorithm for the Classification Problem , 1991 .

[27]  Mo Adam Mahmood,et al.  A PREFORMANCE ANALYSIS OF PARAMETRIC AND NONPARAMETRIC DISCRIMINANT APPROACHES TO BUSINESS DECISION MAKING , 1987 .

[28]  Antonie Stam,et al.  RAGNU: A microcomputer package for two-group mathematical programming-based nonparametric classification , 1995 .

[29]  Patrice Marcotte,et al.  Novel approaches to the discrimination problem , 1992, ZOR Methods Model. Oper. Res..

[30]  J. Ord,et al.  Discriminant Analysis Using Least Absolute Deviations , 1990 .

[31]  William W. Cooper,et al.  Evaluating Program and Managerial Efficiency: An Application of Data Envelopment Analysis to Program Follow Through , 1981 .

[32]  Paul A. Rubin,et al.  Heuristic solution procedures for a mixed‐integer programming discriminant model , 1990 .

[33]  P. Jurs.  Pattern recognition used to investigate multivariate data in analytical chemistry. , 1986, Science.

[34]  Munirpallam A. Venkataramanan,et al.  A genetic algorithm for discriminant analysis , 1998, Ann. Oper. Res..

[35]  Antonie Stam,et al.  A comparison of a robust mixed-integer approach to existing methods for establishing classification rules for the discriminant problem , 1990 .

[36]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[37]  Robert G. Cooper,et al.  The Dimensions of Industrial New Product Success and Failure , 1979 .

[38]  Edward P. Markowski,et al.  An experimental comparison of several approaches to the discriminant problem with both qualitative and quantitative variables , 1987 .

[39]  S. M. Bajgier,et al.  AN EXPERIMENTAL COMPARISON OF STATISTICAL AND LINEAR PROGRAMMING APPROACHES TO THE DISCRIMINANT PROBLEM , 1982 .

[40]  Julius T. Tou,et al.  Pattern Recognition Principles , 1974 .

[41]  R. Kaplan,et al.  Statistical Models of Bond Ratings: A Methodological Inquiry , 1979 .

[42]  Nicola Yanev,et al.  A combinatorial approach to the classification problem , 1999, Eur. J. Oper. Res..

[43]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[44]  Gary J. Koehler,et al.  Minimizing Misclassifications in Linear Discriminant Analysis , 1990 .

[45]  Paul S. Bradley,et al.  Mathematical Programming for Data Mining: Formulations and Challenges , 1999, INFORMS J. Comput..

[46]  John J. Glen,et al.  A comparison of standard and two-stage mathematical programming discriminant analysis methods , 2006, Eur. J. Oper. Res..

[47]  Prakash L. Abad,et al.  New LP based heuristics for the classification problem , 1993 .

[48]  Fred Glover,et al.  Applications and Implementation , 1981 .

[49]  F. Glover,et al.  Simple but powerful goal programming models for discriminant problems , 1981 .

[50]  D. J. Spiegelhalter,et al.  Statistical and Knowledge‐Based Approaches to Clinical Decision‐Support Systems, with an Application in Gastroenterology , 1984 .

[51]  Fred Glover,et al.  IMPROVED LINEAR PROGRAMMING MODELS FOR DISCRIMINANT ANALYSIS , 1990 .