Recommending PSO Variants Using Meta-Learning Framework for Global Optimization

Since inception, particle swarm optimization (PSO) has raised a great interest across various disciplines, thus producing a large number of PSO variants with respective strengths. However, a variant may perform variously on diverse problems, which leads to the risk of the algorithm selection of PSOs for a specific problem without prior knowledge. Hence, it is worth investigating a link between problem characteristics and algorithm performance. To address this issue, we propose a recommendation system of PSO variants for global optimization problem using meta-learning framework. Benchmark functions in the learning instance repository are pictured by meta-features to obtain characteristics and solved by the candidate PSO heuristics to gather performance rankings. k-NN method is employed to develop meta-learning system for recommending the predicted rankings of candidate PSO-variants. Results show that the predicted rankings highly correlate to the ideal rankings and achieve high precision on best algorithm recommendation. Besides, problem surface characteristics play a key role in recommendation performance, followed by sample point characteristics. To sum up, the proposed framework can significantly reduce the risk of algorithm selection.

[1]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[2]  Mario A. Muñoz,et al.  Algorithm selection for black-box continuous optimization problems: A survey on methods and challenges , 2015, Inf. Sci..

[3]  John R. Rice,et al.  The Algorithm Selection Problem , 1976, Adv. Comput..

[4]  Bernd Bischl,et al.  Algorithm selection based on exploratory landscape analysis and cost-sensitive learning , 2012, GECCO '12.

[5]  F. E. Grubbs.  Sample Criteria for Testing Outlying Observations , 1950 .

[6]  Thomas Stützle,et al.  Performance evaluation of automatically tuned continuous optimizers on different benchmark sets , 2015, Appl. Soft Comput..

[7]  L. Darrell Whitley,et al.  The dispersion metric and the CMA evolution strategy , 2006, GECCO.

[8]  Can Cui,et al.  A recommendation system for meta-modeling: A meta-learning based approach , 2016, Expert Syst. Appl..

[9]  Anne Auger,et al.  Performance evaluation of an advanced local search evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[10]  H. Neave.  Distribution-Free Tests , 1988 .

[11]  Bernd Bischl,et al.  Exploratory landscape analysis , 2011, GECCO '11.

[12]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[13]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[14]  Carlos Soares,et al.  Ranking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results , 2003, Machine Learning.

[15]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[16]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[17]  Ricardo Vilalta,et al.  Metalearning - Applications to Data Mining , 2008, Cognitive Technologies.

[18]  Sebastián Ventura,et al.  A meta-learning approach for recommending a subset of white-box classification algorithms for Moodle datasets , 2013, EDM.

[19]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[20]  Terry Jones,et al.  Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms , 1995, ICGA.

[21]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[22]  Kate Smith-Miles,et al.  Towards insightful algorithm selection for optimisation using meta-learning concepts , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[23]  Quan Sun,et al.  Pairwise meta-rules for better meta-learning-based algorithm ranking , 2013, Machine Learning.

[24]  Leandro Nunes de Castro,et al.  Clustering algorithm selection by meta-learning systems: A new distance-based problem characterization and ranking combination methods , 2015, Inf. Sci..

[25]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Meta-learning to select the best meta-heuristic for the Traveling Salesman Problem: A comparison of meta-features , 2016, Neurocomputing.

[26]  Mario A. Muñoz,et al.  A Meta-learning Prediction Model of Algorithm Performance for Continuous Optimization Problems , 2012, PPSN.

[27]  Andries Petrus Engelbrecht,et al.  Particle swarm optimisation failure prediction based on fitness landscape characteristics , 2014, 2014 IEEE Symposium on Swarm Intelligence.

[28]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[29]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..