Determining the significance and relative importance of parameters of a simulated quenching algorithm using statistical tools

When search methods are being designed it is very important to know which parameters have the greatest influence on the behaviour and performance of the algorithm. To this end, algorithm parameters are commonly calibrated by means of either theoretic analysis or intensive experimentation. When undertaking a detailed statistical analysis of the influence of each parameter, the designer should pay attention mostly to the parameters that are statistically significant. In this paper the ANOVA (ANalysis Of the VAriance) method is used to carry out an exhaustive analysis of a simulated annealing based method and the different parameters it requires. Following this idea, the significance and relative importance of the parameters regarding the obtained results, as well as suitable values for each of these, were obtained using ANOVA and post-hoc Tukey HSD test, on four well known function optimization problems and the likelihood function that is used to estimate the parameters involved in the lognormal diffusion process. Through this statistical study we have verified the adequacy of parameter values available in the bibliography using parametric hypothesis tests.

[1]  Dr. Zbigniew Michalewicz,et al.  How to Solve It: Modern Heuristics , 2004 .

[2]  Juan José Rodríguez Diez,et al.  Random projections for linear SVM ensembles , 2011, Applied Intelligence.

[3]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[4]  B. Chakrabarti,et al.  Quantum Annealing and Related Optimization Methods , 2008 .

[5]  Seth D. Guikema,et al.  A derivation of the number of minima of the Griewank function , 2008, Appl. Math. Comput..

[6]  Alex S. Fukunaga,et al.  Distributed island-model genetic algorithms using heterogeneous parameter settings , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[7]  Thomas Stützle,et al.  Automatic Algorithm Configuration Based on Local Search , 2007, AAAI.

[8]  C. J. Kim,et al.  An algorithmic approach for fuzzy inference , 1997, IEEE Trans. Fuzzy Syst..

[9]  A. E. Eiben,et al.  Comparing parameter tuning methods for evolutionary algorithms , 2009, 2009 IEEE Congress on Evolutionary Computation.

[10]  Ajith Abraham,et al.  A Bacterial Evolutionary Algorithm for automatic data clustering , 2009, 2009 IEEE Congress on Evolutionary Computation.

[11]  J. David Schaffer,et al.  An Adaptive Crossover Distribution Mechanism for Genetic Algorithms , 1987, ICGA.

[12]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[13]  L. Darrell Whitley,et al.  Quad Search and Hybrid Genetic Algorithms , 2003, GECCO.

[14]  David B. Fogel,et al.  Evolutionary algorithms in theory and practice , 1997, Complex.

[15]  Xin-She Yang,et al.  Nature-Inspired Metaheuristic Algorithms , 2008 .

[16]  A. E. Eiben,et al.  Parameter Tuning of Evolutionary Algorithms: Generalist vs. Specialist , 2010, EvoApplications.

[17]  Nirwan Ansari,et al.  An efficient annealing algorithm for global optimization in Boltzmann machines , 2004, Applied Intelligence.

[18]  Ramón Gutiérrez,et al.  Inference on some parametric functions in the univeriate lognormal diffusion process with exogenous factors , 2001 .

[19]  A. Griewank Generalized descent for global optimization , 1981 .

[20]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[21]  Nostrand Reinhold,et al.  the utility of using the genetic algorithm approach on the problem of Davis, L. (1991), Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York. , 1991 .

[22]  Ah-Hwee Tan,et al.  On Machine Learning Methods for Chinese Document Categorization , 2003, Applied Intelligence.

[23]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[26]  Lester Ingber,et al.  Simulated annealing: Practice versus theory , 1993 .

[27]  R. Capocelli,et al.  A diffusion model for population growth in random environment. , 1974, Theoretical population biology.

[28]  Jeffrey Horn,et al.  Handbook of evolutionary computation , 1997 .

[29]  A. E. Eiben,et al.  Using Entropy for Parameter Analysis of Evolutionary Algorithms , 2010, Experimental Methods for the Analysis of Optimization Algorithms.

[30]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[31]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[32]  Aimo A. Törn,et al.  Global Optimization , 1999, Science.

[33]  Charles L. Karr,et al.  A Self-Tuning Evolutionary Algorithm Applied to an Inverse Partial Differential Equation , 2003, Applied Intelligence.

[34]  Mark Harman,et al.  Handling dynamic data structures in search based testing , 2008, GECCO '08.

[35]  Chih-Chin Lai,et al.  Special issue on the future and frontier of applied intelligence , 2010, Applied Intelligence.

[36]  Francisco Herrera,et al.  A taxonomy for the crossover operator for real‐coded genetic algorithms: An experimental study , 2003, Int. J. Intell. Syst..

[37]  Christopher R. Stephens,et al.  "Optimal" mutation rates for genetic search , 2006, GECCO.

[38]  Francisco Herrera,et al.  A multi-objective evolutionary algorithm for an effective tuning of fuzzy logic controllers in heating, ventilating and air conditioning systems , 2012, Applied Intelligence.

[39]  Thomas Jansen,et al.  Analysis of evolutionary algorithms for the longest common subsequence problem , 2007, GECCO '07.

[40]  Zbigniew Michalewicz,et al.  Parameter Setting in Evolutionary Algorithms , 2007, Studies in Computational Intelligence.

[41]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[42]  Basel M. Al‐Eideh,et al.  Modelling the CPI using a lognormal diffusion process and implications on forecasting inflation , 2004 .

[43]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[44]  P. Flick,et al.  Evolutionäre Algorithmen , 2012 .

[45]  B. Maddock,et al.  FROM DESIGN TO IMPLEMENTATION , 1982 .

[46]  Zbigniew Michalewicz,et al.  Adaptation in evolutionary computation: a survey , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[47]  Thomas Bartz-Beielstein,et al.  Sequential parameter optimization , 2005, 2005 IEEE Congress on Evolutionary Computation.

[48]  Geoffrey D. Rubin,et al.  Learning-enhanced simulated annealing: method, evaluation, and application to lung nodule registration , 2008, Applied Intelligence.

[49]  Yoav Benjamini,et al.  Opening the Box of a Boxplot , 1988 .

[50]  Thomas Stützle,et al.  A Racing Algorithm for Configuring Metaheuristics , 2002, GECCO.

[51]  Rory A. Fisher,et al.  Theory of Statistical Estimation , 1925, Mathematical Proceedings of the Cambridge Philosophical Society.

[52]  David E. Goldberg,et al.  The Gambler's Ruin Problem, Genetic Algorithms, and the Sizing of Populations , 1999, Evolutionary Computation.

[53]  Xin Yao,et al.  Adapting Self-Adaptive Parameters in Evolutionary Algorithms , 2001, Applied Intelligence.

[54]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[55]  Ilona Jagielska,et al.  An investigation into the application of neural networks, fuzzy logic, genetic algorithms, and rough sets to automated knowledge acquisition for classification problems , 1999, Neurocomputing.

[56]  Héctor Pomares,et al.  Statistical analysis of the main parameters involved in the design of a genetic algorithm , 2002, IEEE Trans. Syst. Man Cybern. Part C.

[57]  Fernando G. Lobo,et al.  A parameter-less genetic algorithm , 1999, GECCO.

[58]  J. Vicente,et al.  Placement by thermodynamic simulated annealing , 2003 .

[59]  Bertrand Neveu,et al.  New requirements for off-line parameter calibration algorithms , 2010, IEEE Congress on Evolutionary Computation.

[60]  Stephanie Forrest,et al.  Proceedings of the 5th International Conference on Genetic Algorithms , 1993 .

[61]  John J. Grefenstette,et al.  Genetic algorithms and their applications , 1987 .

[62]  Dennis Weyland,et al.  Simulated annealing, its parameter settings and the longest common subsequence problem , 2008, GECCO '08.

[63]  A. E. Eiben,et al.  Beating the ‘world champion’ evolutionary algorithm via REVAC tuning , 2010, IEEE Congress on Evolutionary Computation.

[64]  Ignacio Rojas,et al.  Statistical analysis of the parameters of a neuro-genetic algorithm , 2002, IEEE Trans. Neural Networks.

[65]  Laura Diosan,et al.  Improving classification performance of Support Vector Machine by genetically optimising kernel shape and hyper-parameters , 2010, Applied Intelligence.

[66]  Bikas K. Chakrabarti,et al.  Quantum Annealing and Other Optimization Methods , 2005 .

[67]  Risto Miikkulainen,et al.  Measure-theoretic evolutionary annealing , 2011, IEEE Congress on Evolutionary Computation.

[68]  Victor J. Rayward-Smith,et al.  Discretisation of Continuous Commercial Database Features for a Simulated Annealing Data Mining Algorithm , 1999, Applied Intelligence.

[69]  A. E. Eiben,et al.  Efficient relevance estimation and value calibration of evolutionary algorithm parameters , 2007, 2007 IEEE Congress on Evolutionary Computation.

[70]  Dirk Thierens Dimensional Analysis of Allele-Wise Mixing Revisited , 1996, PPSN.

[71]  Sung-Bae Cho,et al.  Exploiting mobile contexts for Petri-net to generate a story in cartoons , 2011, Applied Intelligence.

[72]  Thomas Bartz-Beielstein,et al.  Analysis of Particle Swarm Optimization Using Computational Statistics , 2004 .

[73]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[74]  A. E. Eiben,et al.  A method for parameter calibration and relevance estimation in evolutionary algorithms , 2006, GECCO '06.

[75]  Grahame B. Smith Stuart Geman and Donald Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images”; , 1987 .

[76]  Y. S. Wong,et al.  Development of Heterogeneous Parallel Genetic Simulated Annealing Using Multi-Niche Crowding , 2007 .

[77]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[78]  Erkam Uzun,et al.  A real time traffic simulator utilizing an adaptive fuzzy inference mechanism by tuning fuzzy parameters , 2011, Applied Intelligence.

[79]  Chulhyun Kim,et al.  Forecasting time series with genetic fuzzy predictor ensemble , 1997, IEEE Trans. Fuzzy Syst..

[80]  Thomas Stützle,et al.  Stochastic Local Search , 2007, Handbook of Approximation Algorithms and Metaheuristics.

[81]  Seppo J. Ovaska,et al.  A general framework for statistical performance comparison of evolutionary computation algorithms , 2006, Inf. Sci..

[82]  Miguel Cazorla,et al.  Portable autonomous walk calibration for 4-legged robots , 2010, Applied Intelligence.

[83]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[84]  D. Ackley A connectionist machine for genetic hillclimbing , 1987 .

[85]  Shiu Yin Yuen,et al.  Parameter control by the entire search history: Case study of history-driven evolutionary algorithm , 2010, IEEE Congress on Evolutionary Computation.

[86]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[87]  G. B. Smith,et al.  Preface to S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images” , 1987 .

[88]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (3rd ed.) , 1996 .

[89]  Jürgen Teich,et al.  Systematic integration of parameterized local search into evolutionary algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[90]  Thomas Bartz-Beielstein,et al.  Tuning search algorithms for real-world applications: a regression tree based approach , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[91]  Zbigniew Michalewicz,et al.  Parameter Control in Evolutionary Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[92]  Dirk Thierens,et al.  Toward a Better Understanding of Mixing in Genetic Algorithms , 1993 .