Optimized distance metrics for differential evolution based nearest prototype classifier

In this article, we introduce a differential evolution based classifier with extension for selecting automatically the applied distance measure from a predefined pool of alternative distances measures to suit optimally for classifying the particular data set at hand. The proposed method extends the earlier differential evolution based nearest prototype classifier by extending the optimization process by optimizing not only the required parameters for distance measures, but also optimizing the selection of the distance measure it self in order to find the best possible distance measure for the particular data set at hand. It has been clear for some time that in classification, usual euclidean distance is often not the best choice, and the optimal distance measure depends on the particular properties of the data sets to be classified. So far solving this issue have been subject to a limited attention in the literature. In cases where some consideration to this is problem is given, there has only been testing with couple distance measure to find which one applies best to the data at hand. In this paper we have attempted to take one step further by applying a systematic global optimization approach for selecting the best distance measure from a set of alternative measures for obtaining the highest classification accuracy for the given data. In particular, we have generated pool of distance measures for the purpose and developed a model on how the differential evolution based classifier can be extended to optimize the selection of the distance measure for given data. The obtained results are demonstrating, and also confirming further on the earlier findings reported in the literature, that often some other distance measure than the most commonly used euclidean distance is the best choice. The selection of distance measure is one of the most important factor for obtaining best classification accuracy, and should thereby be emphasized more in future research. The results also indicate that it is possible to build a classifier that is selecting the optimal distance measure for the given data automatically. It is also recommended that the proposed extension the differential evolution based classifier is clearly efficient alternative in solving classification problems.

[1]  Stefania Bertazzon,et al.  Comparison of distance measures in spatial analytical modeling for health service planning , 2009, BMC health services research.

[2]  Hans-Jürgen Zimmermann,et al.  Fuzzy Data Analysis , 1996 .

[3]  John H. Holland,et al.  Properties of the bucket brigade algorithm , 1985 .

[4]  A. Suruliandi,et al.  Empirical evaluation of distance measures for supervised classification of remotely sensed image with Modified Multivariate Local Binary Pattern , 2011, 2011 International Conference on Emerging Trends in Electrical and Computer Technology.

[5]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[6]  E. Dettmann,et al.  Distance functions for matching in small samples , 2011, Comput. Stat. Data Anal..

[7]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[8]  W. T. Penzhorn,et al.  Experimental comparison on several distance measures for speech processing applications , 1988, COMSIG 88@m_Southern African Conference on Communications and Signal Processing. Proceedings.

[9]  Terence C. Fogarty,et al.  Co-Evolving Co-Operative Populations of Rules in Learning Control Systems , 1994, Evolutionary Computing, AISB Workshop.

[10]  Adel Al-Jumaily,et al.  Feature subset selection using differential evolution and a statistical repair mechanism , 2011, Expert Syst. Appl..

[11]  Enis Günay,et al.  Efficient edge detection in digital images using a cellular neural network optimized by differential evolution algorithm , 2009, Expert Syst. Appl..

[12]  Fernando Fernández,et al.  Evolutionary Design of Nearest Prototype Classifiers , 2004, J. Heuristics.

[13]  Daniel A. Ashlock,et al.  Evaluating Distance Measures for RNA Motif Search , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[14]  Jouni Lampinen,et al.  Differential Evolution Classifier in Noisy Settings and with Interacting Variables , 2011, Appl. Soft Comput..

[15]  Yang Liu,et al.  A fast differential evolution algorithm using k-Nearest Neighbour predictor , 2011, Expert Syst. Appl..

[16]  Satoshi Takahashi,et al.  Weighted distance measures for efficient reduction of Gaussian mixture components in HMM-based acoustic model , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Dejan Gjorgjevikj,et al.  Evaluation of Distance Measures for Multi-class Classification in Binary SVM Decision Tree , 2010, ICAISC.

[18]  John H. Holland,et al.  Properties of the Bucket Brigade , 1985, ICGA.

[19]  George G. Robertson,et al.  Parallel Implementation of Genetic Algorithms in a Classifier Rystem , 1987, ICGA.

[20]  Jun Zhang,et al.  Comparison of Distance Measures in Evolutionary Time Series Segmentation , 2007, Third International Conference on Natural Computation (ICNC 2007).

[21]  Chi-Keong Goh,et al.  Computational Intelligence in Optimization: Applications and Implementations , 2010 .

[22]  Samuel Kaski,et al.  Bankruptcy analysis with self-organizing maps in learning metrics , 2001, IEEE Trans. Neural Networks.

[23]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[24]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[25]  Stewart W. Wilson.  Hierarchical Credit Allocation in a Classifier System , 1987, IJCAI.

[26]  John H. Holland.  Genetic Algorithms and Classifier Systems: Foundations and Future Directions , 1987, ICGA.

[27]  Jouni Lampinen,et al.  A Classification method based on principal component analysis and differential evolution algorithm applied for prediction diagnosis from clinical EMR heart data sets , 2010 .

[28]  Lashon B. Booker,et al.  Improving the Performance of Genetic Algorithms in Classifier Systems , 1985, ICGA.

[29]  Riccardo Poli,et al.  New ideas in optimization , 1999 .

[30]  Liang Zhao,et al.  Classification rule discovery with DE/QDE algorithm , 2010, Expert Syst. Appl..

[31]  Hong Chang,et al.  Relaxational metric adaptation and its application to semi-supervised clustering and content-based image retrieval , 2006, Pattern Recognit..

[32]  Vadlamani Ravi,et al.  Differential evolution trained wavelet neural networks: Application to bankruptcy prediction in banks , 2009, Expert Syst. Appl..