Statistical analysis of convergence performance throughout the evolutionary search: A case study with SaDE-MMTS and Sa-EPSDE-MMTS

Typically, comparisons among optimization algorithms only considers the results obtained at the end of the search process. However, there are occasions in which is very interesting to perform comparisons along the search. This way, algorithms could also be categorized depending on its convergence performance, which would help when deciding which algorithms perform better among a set of methods that are assumed as equal when only the results at the end of the search are considered. In this work, we present a procedure to perform a pairwise comparison of two algorithms' convergence performance. A non-parametric procedure, the Page test, is used to detect significant differences between the evolution of the error of the algorithms as the search continues. A case of study has been also provided to demonstrate the application of the test.

[1]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[2]  Mehmet Fatih Tasgetiren,et al.  Differential evolution algorithm with ensemble of parameters and mutation strategies , 2011, Appl. Soft Comput..

[3]  A. Kai Qin,et al.  Self-adaptive differential evolution algorithm for numerical optimization , 2005, 2005 IEEE Congress on Evolutionary Computation.

[4]  Ponnuthurai N. Suganthan,et al.  Self-adaptive differential evolution with multi-trajectory search for large-scale optimization , 2011, Soft Comput..

[5]  E. B. Page.  Ordered Hypotheses for Multiple Treatments: A Significance Test for Linear Ranks , 1963 .

[6]  Francisco Herrera,et al.  A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability , 2009, Soft Comput..

[7]  Ponnuthurai N. Suganthan,et al.  Comprehensive comparison of convergence performance of optimization algorithms based on nonparametric statistical tests , 2012, 2012 IEEE Congress on Evolutionary Computation.

[8]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[9]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[10]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[11]  Mario Cortina-Borja,et al.  Handbook of Parametric and Nonparametric Statistical Procedures, 5th edn , 2012 .

[12]  Jean Dickinson Gibbons,et al.  Nonparametric Statistical Inference. 2nd Edition. , 1986 .

[13]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[14]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[15]  D. Sheskin.  Handbook of Parametric and Nonparametric Statistical Procedures: Third Edition , 2000 .