Differential Evolution Classifier in Noisy Settings and with Interacting Variables

In this paper, we have studied the performance of a differential evolution (DE) classifier in classifying data in noisy settings. We have also studied the performance in handling extra variables which simply consists of gaussian noise. Furthermore, we have carried out the classification by adding on all two component interaction terms as extra variables into the data. Also, in this situation it is crucial to have a classifier which is tolerant to noisy variables. Namely, even though there can be interaction effects in the data that can influence classification results positively, it is usually not known a priori which particular interaction components are contributing to the classification results. Therefore, we need to add all possible combinations despite the likelihood of then creating also some noisy variables which do not influence the classification accuracy, or which actually reduce the accuracy. In experimentation, we used four widely applied test data sets; the new-thyroid, heart-statlog, Hungarian heart and lenses data sets. The results indicated the DE classifier to be robust from the noise tolerance point of view in all studied cases and situations. The results suggest that the DE classifier is useful especially in the cases where interaction effects may have a significant influence to the classification accuracy.

[1]  Kenneth V. Price,et al.  An introduction to differential evolution , 1999 .

[2]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[3]  B. Bowerman.  Statistical Design and Analysis of Experiments, with Applications to Engineering and Science , 1989 .

[4]  Robert P. W. Duin,et al.  A Matlab Toolbox for Pattern Recognition , 2004 .

[5]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[6]  Aik Choon Tan,et al.  Ensemble machine learning on gene expression data for cancer classification. , 2003, Applied bioinformatics.

[7]  J. J. Chen,et al.  Classification ensembles for unbalanced class sizes in predictive toxicology , 2005, SAR and QSAR in environmental research.

[8]  Loris Nanni,et al.  Particle swarm optimization for prototype reduction , 2009, Neurocomputing.

[9]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[10]  J. S. Hunter,et al.  Statistics for Experimenters: Design, Innovation, and Discovery , 2006 .

[11]  John H. Holland,et al.  Properties of the Bucket Brigade , 1985, ICGA.

[12]  Inés María Galván,et al.  An Adaptive Michigan Approach PSO for Nearest Prototype Classification , 2007, IWINAC.

[13]  Terence C. Fogarty,et al.  Co-Evolving Co-Operative Populations of Rules in Learning Control Systems , 1994, Evolutionary Computing, AISB Workshop.

[14]  Stewart W. Wilson.  Hierarchical Credit Allocation in a Classifier System , 1987, IJCAI.

[15]  Lashon B. Booker,et al.  Improving the Performance of Genetic Algorithms in Classifier Systems , 1985, ICGA.

[16]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[17]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[18]  Wojtek J. Krzanowski,et al.  Principles of multivariate analysis : a user's perspective. oxford , 1988 .

[19]  Brian Everitt,et al.  Principles of Multivariate Analysis , 2001 .

[20]  Fernando Fernández,et al.  Evolutionary Design of Nearest Prototype Classifiers , 2004, J. Heuristics.

[21]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[22]  Riccardo Poli,et al.  New ideas in optimization , 1999 .

[23]  D. Coomans,et al.  Comparison of Multivariate Discrimination Techniques for Clinical Data— Application to the Thyroid Functional State , 1983, Methods of Information in Medicine.

[24]  John H. Holland,et al.  Properties of the bucket brigade algorithm , 1985 .

[25]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[26]  Jouni Lampinen,et al.  A Classification method based on principal component analysis and differential evolution algorithm applied for prediction diagnosis from clinical EMR heart data sets , 2010 .

[27]  Chi-Keong Goh,et al.  Computational Intelligence in Optimization: Applications and Implementations , 2010 .

[28]  Philip M. Long,et al.  Boosting and Microarray Data , 2003, Machine Learning.

[29]  Ivanoe De Falco,et al.  Facing classification problems with Particle Swarm Optimization , 2007, Appl. Soft Comput..

[30]  Joseph G. Pigeon,et al.  Statistics for Experimenters: Design, Innovation and Discovery , 2006, Technometrics.

[31]  George G. Robertson,et al.  Parallel Implementation of Genetic Algorithms in a Classifier Rystem , 1987, ICGA.

[32]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..