Variable Neighborhood Programming For Evolving Discriminent Functions With Dynamic Thresholds

This paper aims to develop a new automatic programming system to solve multi-class classification problem. Our method is based on a recently proposed algorithm called variable neighborhood programming (VNP). VNP is a solution-based metaheuristic which evolves programs and classifiers. Numeric expression is one of classifier representation in automatic programming algorithms. In the case of binary classification, a translation can simply performed according to the sign of the output (negative: class 1, positive: class 2). However, when the number of classes is higher, setting the appropriate boundary values and fixing the optimal class order is more complicated. To solve this problem, we introduce a new local search which aims to find dynamically the more adequate thresholds for each classifier. To include the developed local search within VNP (and also within any automatic programming algorithm), a multidimensional schema is proposed to ensure the simultaneous optimization of the discriminant function (tree),the class order, and the corresponding boundaries. Experimental results on three real data sets show that the proposed approach is a good automatic programming tool, able to build a full classifier with high accuracy in a single run.

[1]  Daniel Howard,et al.  Target detection in SAR imagery by genetic programming , 1999 .

[2]  Abdul Rauf Baig,et al.  Two-stage learning for multi-class classification using genetic programming , 2013, Neurocomputing.

[3]  Chuang Deng,et al.  A Novel Multiclass Classification Method with Gene Expression Programming , 2009, 2009 International Conference on Web Information Systems and Mining.

[4]  Hasan Bal,et al.  Comparing performances of backpropagation and genetic algorithms in the data classification , 2011, Expert Syst. Appl..

[5]  Luis Fernando de Mingo López,et al.  The optimal combination: Grammatical swarm, particle swarm optimization and neural networks , 2012, J. Comput. Sci..

[6]  Gérard Ramstein,et al.  A Grammatical Swarm for protein classification , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[7]  Walter Alden Tackett,et al.  Genetic Programming for Feature Discovery and Image Discrimination , 1993, ICGA.

[8]  John R. Koza,et al.  Genetic programming as a means for programming computers by natural selection , 1994 .

[9]  Mengjie Zhang,et al.  Multiclass Object Classification Using Genetic Programming , 2004, EvoWorkshops.

[10]  Pei-Chann Chang,et al.  Data Clustering and Evolving Fuzzy Decision Tree for Data Base Classification Problems , 2008, ICIC.

[11]  Victor Ciesielski,et al.  A Domain-Independent Window Approach to Multiclass Object Detection Using Genetic Programming , 2003, EURASIP J. Adv. Signal Process..

[12]  Li Zhang,et al.  Hybrid decision tree and naïve Bayes classifiers for multi-class classification tasks , 2014, Expert Syst. Appl..

[13]  Sebastián Ventura,et al.  Parallel multi-objective Ant Programming for classification using GPUs , 2013, J. Parallel Distributed Comput..

[14]  Nenad Mladenović,et al.  Variable neighborhood programming - A new automatic programming method in artificial intelligence , 2016 .

[15]  Cândida Ferreira,et al.  Gene Expression Programming: A New Adaptive Algorithm for Solving Problems , 2001, Complex Syst..

[16]  Pierre Hansen,et al.  New VNP for automatic programming , 2017, Electron. Notes Discret. Math..

[17]  Sebastián Ventura,et al.  Using Ant Programming Guided by Grammar for Building Rule-Based Classifiers , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Michael O'Neill,et al.  Grammatical Evolution: Evolving Programs for an Arbitrary Language , 1998, EuroGP.

[19]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[20]  D.C. Hope,et al.  Evolutionary Algorithms in the Classification of Mammograms , 2007, 2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing.

[21]  Pierre Hansen,et al.  Variable neighborhood search , 1997, Eur. J. Oper. Res..

[22]  Vic Ciesielski,et al.  Texture classifiers generated by genetic programming , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[23]  Sebastián Ventura,et al.  Classification rule mining using ant programming guided by grammar with multiple Pareto fronts , 2012, Soft Comput..

[24]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[25]  Victor Ciesielski,et al.  Genetic Programming for Multiple Class Object Detection , 1999, Australian Joint Conference on Artificial Intelligence.

[26]  Nikhil R. Pal,et al.  A Multiobjective Genetic Programming-Based Ensemble for Simultaneous Feature Selection and Classification , 2016, IEEE Transactions on Cybernetics.