A mixed-integer programming approach to multi-class data classification problem

This paper presents a new data classification method based on mixed-integer programming. Traditional approaches that are based on partitioning the data sets into two groups perform poorly for multi-class data classification problems. The proposed approach is based on the use of hyper-boxes for defining boundaries of the classes that include all or some of the points in that set. A mixed-integer programming model is developed for representing existence of hyper-boxes and their boundaries. In addition, the relationships among the discrete decisions in the model are represented using propositional logic and then converted to their equivalent integer constraints using Boolean algebra. The proposed approach for multi-class data classification is illustrated on an example problem. The efficiency of the proposed method is tested on the well-known IRIS data set. The computational results on the illustrative example and the IRIS data set show that the proposed method is very accurate and efficient on multi-class data classification problems.

[1]  David Kendrick,et al.  GAMS, a user's guide , 1988, SGNM.

[2]  S. Selcuk Erenguc,et al.  Survey of mathematical programming models and experimental results for linear discriminant analysis , 1990 .

[3]  Philip S. Yu,et al.  Data Mining: An Overview from a Database Perspective , 1996, IEEE Trans. Knowl. Data Eng..

[4]  Tzung-Pei Hong,et al.  Induction of fuzzy rules and membership functions from training examples , 1996, Fuzzy Sets Syst..

[5]  Yasutoshi Yajima,et al.  Linear programming approaches for multicategory support vector machines , 2005, Eur. J. Oper. Res..

[6]  C. Zhang,et al.  Prediction of protein (domain) structural classes based on amino-acid index. , 1999, European journal of biochemistry.

[7]  Yu-Dong Cai,et al.  Support Vector Machines for predicting protein structural class , 2001, BMC Bioinformatics.

[8]  Andrzej Skowron,et al.  Discovery of Data Patterns with Applications to Decomposition and Classification Problems , 1998 .

[9]  Bernhard Schölkopf,et al.  Improving the accuracy and speed of support vector learning machines , 1997, NIPS 1997.

[10]  Hyun-Chul Kim,et al.  Constructing support vector machine ensemble , 2003, Pattern Recognit..

[11]  Panos M. Pardalos,et al.  Modeling and integer programming techniques applied to propositional calculus , 1990, Comput. Oper. Res..

[12]  S. M. Bajgier,et al.  AN EXPERIMENTAL COMPARISON OF STATISTICAL AND LINEAR PROGRAMMING APPROACHES TO THE DISCRIMINANT PROBLEM , 1982 .

[13]  P Argos,et al.  Prediction of secondary structural content of proteins from their amino acid composition alone. II. The paradox with secondary structural class , 1996, Proteins.

[14]  Herb Edelstein,et al.  Building profitable customer relationships with data mining , 2000 .

[15]  Chin-Chuan Han,et al.  A modular eigen subspace scheme for high-dimensional data classification , 2004, Future Gener. Comput. Syst..

[16]  P. K. Simpson.  Fuzzy Min-Max Neural Networks-Part 1 : Classification , 1992 .

[17]  Richard J. Roiger,et al.  Data Mining: A Tutorial Based Primer , 2002 .

[18]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[19]  Willy Gochet,et al.  Mathematical programming based heuristics for improving LP-generated classifiers for the multiclass supervised classification problem , 2006, Eur. J. Oper. Res..

[20]  K. Chou,et al.  Prediction and classification of domain structural classes , 1998, Proteins.

[21]  G. McLachlan.  Discriminant Analysis and Statistical Pattern Recognition , 1992 .

[22]  Ignacio E. Grossmann,et al.  Disjunctive Programming Techniques for the Optimization of Process Systems with Discontinuous Investment Costs−Multiple Size Regions , 1996 .

[23]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[24]  Robert P. W. Duin,et al.  Using two-class classifiers for multiclass classification , 2002, Object recognition supported by user interaction for service robots.

[25]  Chin-Teng Lin,et al.  Neural-Network-Based Fuzzy Logic Control and Decision System , 1991, IEEE Trans. Computers.

[26]  David J. Hand,et al.  Discrimination and Classification , 1982 .

[27]  Bernhard Schölkopf,et al.  Extracting Support Data for a Given Task , 1995, KDD.

[28]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[29]  Yu-Dong Cai,et al.  Is it a paradox or misinterpretation? , 2001, Proteins.

[30]  Antonie Stam,et al.  A comparison of a robust mixed-integer approach to existing methods for establishing classification rules for the discriminant problem , 1990 .

[31]  K. Chou,et al.  Does the folding type of a protein depend on its amino acid composition? , 1995, FEBS letters.

[32]  Jose Jesus Castro-Schez,et al.  Learning maximal structure rules in fuzzy logic for knowledge acquisition in expert systems , 1999, Fuzzy Sets Syst..

[33]  R. Jernigan,et al.  Understanding the recognition of protein structural classes by amino acid composition , 1997, Proteins.

[34]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[35]  M. R. Mickey,et al.  Estimation of Error Rates in Discriminant Analysis , 1968 .

[36]  John M. Liittschwager,et al.  Integer Programming Solution of a Classification Problem , 1978 .

[37]  R. Fisher.  The Advanced Theory of Statistics , 1943, Nature.

[38]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[39]  P Argos,et al.  Prediction of secondary structural content of proteins from their amino acid composition alone. I. New analytic vector decomposition methods , 1996, Proteins.

[40]  W. Gehrlein.  General mathematical programming formulations for the statistical classification problem , 1986 .

[41]  Shyi-Ming Chen,et al.  A new method for constructing membership functions and fuzzy rules from training examples , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[42]  Shyi-Ming Chen,et al.  A New Approach for Handling the Iris Data Classification Problem , 2005 .

[43]  Toshihide Ibaraki,et al.  Logical analysis of numerical data , 1997, Math. Program..

[44]  Gary J. Koehler,et al.  Considerations for mathematical programming models in discriminant analysis , 1990 .

[45]  R. Raman,et al.  RELATION BETWEEN MILP MODELLING AND LOGICAL INFERENCE FOR CHEMICAL PROCESS SYNTHESIS , 1991 .

[46]  Daijin Kim,et al.  Data classification based on tolerant rough set , 2001, Pattern Recognit..

[47]  김대진,et al.  허용적 러프 집합에 기반한 데이터 분류 ( Data Classification based on Tolerant Rough Set ) , 1997 .

[48]  E A Joachimsthaler,et al.  Mathematical Programming Approaches for the Classification Problem in Two-Group Discriminant Analysis. , 1990, Multivariate behavioral research.

[49]  K Nishikawa,et al.  The folding type of a protein is relevant to the amino acid composition. , 1986, Journal of biochemistry.

[50]  Guo-Ping Zhou,et al.  An Intriguing Controversy over Protein Structural Class Prediction , 1998, Journal of protein chemistry.

[51]  Martin Anthony,et al.  On data classification by iterative linear partitioning , 2004, Discret. Appl. Math..

[52]  C. Chothia,et al.  Structural patterns in globular proteins , 1976, Nature.

[53]  Y Cai,et al.  Prediction of protein structural classes by neural network. , 2000, Biochimie.

[54]  Casimir A. Kulikowski,et al.  Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning and Expert Systems , 1990 .