Stochastic algorithms assessment using performance profiles

Optimization with stochastic algorithms has become a relevant approach, specially, in problems with complex search spaces. Due to the stochastic nature of these algorithms, the assessment and comparison is not straightforward. Several performance measures have been proposed to overcome this difficulty. In this work, the use of performance profiles and an analysis integrating a trade-off between accuracy and precision are carried out for the comparison of two stochastic algorithms. Traditionally, performance profiles are used to compare deterministic algorithms. This methodology is applied in the comparison of two stochastic algorithms - genetic algorithms and simulated annealing. The results highlight the advantages and drawbacks of the proposed assessment.

[1]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[2]  Zelda B. Zabinsky,et al.  A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems , 2005, J. Glob. Optim..

[3]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[4]  Michael N. Vrahatis,et al.  Memetic particle swarm optimization , 2007, Ann. Oper. Res..

[5]  André da Motta Salles Barreto,et al.  Probabilistic performance profiles for the experimental evaluation of stochastic algorithms , 2010, GECCO '10.

[6]  Ritu Gupta,et al.  Statistical exploratory analysis of genetic algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[7]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[8]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[9]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[10]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[11]  Lester Ingber,et al.  Adaptive simulated annealing (ASA): Lessons learned , 2000, ArXiv.

[12]  Héctor Pomares,et al.  Statistical analysis of the main parameters involved in the design of a genetic algorithm , 2002, IEEE Trans. Syst. Man Cybern. Part C.