Parallel processing for dynamic multi-objetive optimization

The main objective of this PhD thesis is to advance the field of parallel multi-objective evolutionary algorithms to solve dynamic multi-objective optimization problems, Thus, the research presented in this thesis involves three different, although related, fields: - Multi-objective evolutionary algorithms (MOEA), - Dynamic multi-objective optimization (DMO) problems, and - Parallelization of MOEAs to solve DMO problems. The degree of advancement of the research varies for each of the afore-mentioned topics, from a full-fledged research field as it is the MOEA topic to a new emerging subject as it happens with dynamic multi-objective optimization. Nevertheless, proposals to improve further the three afore-mentioned subjects have been made in this thesis. First of all, this thesis introduces a \textit{low-cost} MOEA able to deal with multi-objective problems within more restrictive time limits than other state-of-the-art can do. Secondly, the field of dynamic optimization is reviewed and some additions are made so that the field moves forward to tackle dynamic multi-objective problems. This has been facilitated by the introduction of performance measures for problems that are both dynamic and multi-objective. Moreover, modifications are proposed for two of the five \textit{de facto} standard test cases for DMO problems. Thirdly, the parallelization of MOEAs to solve DMO problems is addressed with two different proposed approaches: - A hybrid master-worker and island approach called pdMOEA, and - A fully distributed approach called pdMOEA+. These two approaches are compared side-by-side with the test cases already mentioned. Finally, future work to follow upon the achievements of this thesis is outlined.

[1]  Julio Ortega Lopera,et al.  Parallel Processing for Multi-objective Optimization in Dynamic Environments , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[2]  Peter Seibel,et al.  Practical Common Lisp , 2005 .

[3]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[4]  Enrique Alba,et al.  Parallel Evolutionary Multiobjective Optimization , 2006, Parallel Evolutionary Computations.

[5]  C. Reeves.  Modern heuristic techniques for combinatorial problems , 1993 .

[6]  Kalyanmoy Deb,et al.  Distributed computing of Pareto-optimal solutions using multi-objective evolutionary algorithms , 2003 .

[7]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[8]  Nils Aall Barricelli,et al.  Numerical testing of evolution theories , 1963 .

[9]  Kalyanmoy Deb,et al.  Simulated Binary Crossover for Continuous Search Space , 1995, Complex Syst..

[10]  Marco Laumanns,et al.  SPEA2: Improving the Strength Pareto Evolutionary Algorithm For Multiobjective Optimization , 2002 .

[11]  Wan-Suk Yoo,et al.  Multi-objective optimization of tire carcass contours using a systematic aspiration-level adjustment procedure , 2002 .

[12]  Samir Saoudi,et al.  Stochastic K-means algorithm for vector quantization , 2001, Pattern Recognit. Lett..

[13]  Günter Rudolph,et al.  Evolutionary Optimization of Dynamic Multiobjective Functions , 2006 .

[14]  李幼升,et al.  Ph , 1989 .

[15]  David H. Wolpert,et al.  Coevolutionary free lunches , 2005, IEEE Transactions on Evolutionary Computation.

[16]  Mario Köppen.  No-Free-Lunch theorems and the diversity of algorithms , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[17]  David E. Goldberg,et al.  Nonstationary Function Optimization Using Genetic Algorithms with Dominance and Diploidy , 1987, ICGA.

[18]  Marco Laumanns,et al.  Evolutionary Multiobjective Design in Automotive Development , 2005, Applied Intelligence.

[19]  Anthony Brabazon,et al.  Foundations in Grammatical Evolution for Dynamic Environments , 2009, Studies in Computational Intelligence.

[20]  Hussein A. Abbass,et al.  Local models—an approach to distributed multi-objective optimization , 2009, Comput. Optim. Appl..

[21]  M. Chowdhury,et al.  Benchmarks for testing evolutionary algorithms , 1998 .

[22]  Shengxiang Yang,et al.  Genetic Algorithms with Memory- and Elitism-Based Immigrants in Dynamic Environments , 2008, Evolutionary Computation.

[23]  Enrique Alba Torres.  Análisis y diseño de algoritmos genéticos paralelos distribuidos , 1999 .

[24]  Lam Thu Bui.  The role of communication messages andexplicit niching in distributed evolutionarymulti-objective optimization , 2007 .

[25]  Carlos A. Coello Coello,et al.  Advances in Multi-Objective Nature Inspired Computing , 2010, Advances in Multi-Objective Nature Inspired Computing.

[26]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[27]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[28]  Kalyanmoy Deb,et al.  Dynamic Multi-objective Optimization and Decision-Making Using Modified NSGA-II: A Case Study on Hydro-thermal Power Scheduling , 2007, EMO.

[29]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[30]  Jürgen Branke,et al.  Proceedings of the Workshop on Evolutionary Algorithms for Dynamic Optimization Problems (EvoDOP-2003) held in conjunction with the Genetic and Evolutionary Computation Conference (GECCO-2003), 12 July 2003, Chicago, USA [online] , 2003 .

[31]  Peter A. N. Bosman,et al.  Learning, anticipation and time-deception in evolutionary online dynamic optimization , 2005, GECCO '05.

[32]  Marco Laumanns,et al.  Why Quality Assessment Of Multiobjective Optimizers Is Difficult , 2002, GECCO.

[33]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[34]  Hajime Kita,et al.  Adaptation to a Changing Environment by Means of the Feedback Thermodynamical Genetic Algorithm , 1996, PPSN.

[35]  Ralf Salomon,et al.  Adaptation on the Evolutionary Time Scale: A Working Hypothesis and Basic Experiments , 1997, Artificial Evolution.

[36]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[37]  Francisco Fernndez de Vega,et al.  Parallel and Distributed Computational Intelligence , 2010, Parallel and Distributed Computational Intelligence.

[38]  Appa Iyer Sivakumar,et al.  Pareto Control in Multi-Objective Dynamic Scheduling of a Stepper Machine in Semiconductor Wafer Fabrication , 2006, Proceedings of the 2006 Winter Simulation Conference.

[39]  F. de Toro,et al.  PSFGA: a parallel genetic algorithm for multiobjective optimization , 2002, Proceedings 10th Euromicro Workshop on Parallel, Distributed and Network-based Processing.

[40]  Jürgen Branke,et al.  Evolutionary Optimization in Dynamic Environments , 2001, Genetic Algorithms and Evolutionary Computation.

[41]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[42]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[43]  Peter J. Fleming,et al.  On the Performance Assessment and Comparison of Stochastic Multiobjective Optimizers , 1996, PPSN.

[44]  Kalyanmoy Deb,et al.  Multiobjective Problem Solving from Nature: From Concepts to Applications , 2008, Natural Computing Series.

[45]  Maarten Keijzer,et al.  Evolving Objects: A General Purpose Evolutionary Computation Library , 2001, Artificial Evolution.

[46]  Yuping Wang,et al.  New Evolutionary Algorithm for Dynamic Multiobjective Optimization Problems , 2006, ICNC.

[47]  Shigeyoshi Tsutsui,et al.  Advances in evolutionary computing: theory and applications , 2003 .

[48]  Enrique Alba,et al.  A cellular multi-objective genetic algorithm for optimal broadcasting strategy in metropolitan MANETs , 2007, Comput. Commun..

[49]  Conor Ryan,et al.  Grammatical Evolution , 2001, Genetic Programming Series.

[50]  Ben Paechter,et al.  A Hybrid Meta-Heuristic for Multi-Objective Optimization: MOSATS , 2007, J. Math. Model. Algorithms.

[51]  Xin Yao,et al.  Empirical analysis of evolutionary algorithms with immigrants schemes for dynamic optimization , 2009, Memetic Comput..

[52]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[53]  E. Talbi.  Parallel combinatorial optimization , 2006 .

[54]  Victor J. Katz.  The history of mathematics , 1992 .

[55]  Jürgen Branke,et al.  Memory enhanced evolutionary algorithms for changing optimization problems , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[56]  Jeffrey Horn,et al.  Multiobjective Optimization Using the Niched Pareto Genetic Algorithm , 1993 .

[57]  Zbigniew Michalewicz,et al.  Searching for optima in non-stationary environments , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[58]  Enrique Alba,et al.  Parallel evolutionary algorithms can achieve super-linear performance , 2002, Inf. Process. Lett..

[59]  Jürgen Branke,et al.  A Multi-population Approach to Dynamic Optimization Problems , 2000 .

[60]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[61]  Erick Cantú-Paz,et al.  Efficient and Accurate Parallel Genetic Algorithms , 2000, Genetic Algorithms and Evolutionary Computation.

[62]  Jiebo Luo,et al.  Image segmentation via adaptive K-mean clustering and knowledge-based morphological operations with biomedical applications , 1998, IEEE Trans. Image Process..

[63]  Loo Hay Lee,et al.  Application of multi-objective simulation-optimization techniques to inventory management problems , 2005, Proceedings of the Winter Simulation Conference, 2005..

[64]  John R. Koza,et al.  Genetic Programming II , 1992 .

[65]  Enrique Alba,et al.  Análisis y diseño de algoritmos genéticos paralelos distribuidos , 2011 .

[66]  Shengxiang Yang,et al.  Evolutionary Computation in Dynamic and Uncertain Environments (Studies in Computational Intelligence) , 2007 .

[67]  Enrique Alba,et al.  Parallel Evolutionary Computations , 2006, Studies in Computational Intelligence.

[68]  Peter A. N. Bosman,et al.  Learning and anticipation in online dynamic optimization with evolutionary algorithms: the stochastic case , 2007, GECCO '07.

[69]  D. Dasgupta.  Artificial Immune Systems and Their Applications , 1998, Springer Berlin Heidelberg.

[70]  Lawrence J. Fogel,et al.  Intelligence Through Simulated Evolution: Forty Years of Evolutionary Programming , 1999 .

[71]  A. Sima Etaner-Uyar,et al.  Towards an analysis of dynamic environments , 2005, GECCO '05.

[72]  Christos H. Papadimitriou,et al.  Logicomix: An Epic Search for Truth , 2008 .

[73]  Claudio Rossi,et al.  Tracking Moving Optima Using Kalman-Based Predictions , 2008, Evolutionary Computation.

[74]  M. Koishi,et al.  Multi-Objective Design Problem of Tire Wear and Visualization of Its Pareto Solutions2 , 2006 .

[75]  Hajime Kita,et al.  Adaption to a Changing Environment by Means of the Thermodynamical Genetic Algorithm , 1996, PPSN.

[76]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[77]  Francisco de Toro,et al.  The Parallel Single Front Genetic Algorithm (PSFGA) in Dynamic Multi-objective Optimization , 2007, IWANN.

[78]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[79]  Ben Paechter,et al.  PSFGA : Parallel processing and evolutionary computation for multiobjective optimisation , 2004 .

[80]  D.A. Van Veldhuizen,et al.  On measuring multiobjective evolutionary algorithm performance , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[81]  Gary B. Lamont,et al.  Considerations in engineering parallel multiobjective evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[82]  David W. Corne,et al.  No Free Lunch and Free Leftovers Theorems for Multiobjective Optimisation Problems , 2003, EMO.

[83]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[84]  Francisco Luna,et al.  Solving large-scale real-world telecommunication problems using a grid-based genetic algorithm , 2008 .

[85]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[86]  Massimiliano Gobbi,et al.  Evolutionary multiobjective industrial design: the case of a racing car tire-suspension system , 2006, IEEE Transactions on Evolutionary Computation.

[87]  Lothar Thiele,et al.  A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers , 2006 .

[88]  Shengxiang Yang,et al.  A self-organizing random immigrants genetic algorithm for dynamic optimization problems , 2007, Genetic Programming and Evolvable Machines.

[89]  Piotr Czyzżak,et al.  Pareto simulated annealing—a metaheuristic technique for multiple‐objective combinatorial optimization , 1998 .

[90]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[91]  Jürgen Branke *,et al.  Anticipation and flexibility in dynamic scheduling , 2005 .

[92]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[93]  Francisco de Toro,et al.  High performance computing for dynamic multi-objective optimisation , 2008, Int. J. High Perform. Syst. Archit..

[94]  Terence C. Fogarty,et al.  A Genetic Algorithm with Variable Range of Local Search for Tracking Changing Environments , 1996, PPSN.

[95]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[96]  John J. Grefenstette,et al.  Genetic algorithms and their applications , 1987 .

[97]  Hajime Kita,et al.  Adaptation to a Changing Environment by Means of the Thermodynamical Genetic Algorithm , 1999 .

[98]  Anne Auger,et al.  Theory of the hypervolume indicator: optimal μ-distributions and the choice of the reference point , 2009, FOGA '09.

[99]  David Corne,et al.  The Pareto archived evolution strategy: a new baseline algorithm for Pareto multiobjective optimisation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[100]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .

[101]  Andreas Zell,et al.  Parallelization of Multi-objective Evolutionary Algorithms Using Clustering Algorithms , 2005, EMO.

[102]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[103]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[104]  El-Ghazali Talbi,et al.  ParadisEO: A Framework for the Reusable Design of Parallel and Distributed Metaheuristics , 2004, J. Heuristics.

[105]  Victor J. Katz,et al.  A History of Mathematics: An Introduction , 1998 .

[106]  Karsten Weicker,et al.  Performance Measures for Dynamic Environments , 2002, PPSN.

[107]  Kalyanmoy Deb,et al.  Dynamic multiobjective optimization problems: test cases, approximations, and applications , 2004, IEEE Transactions on Evolutionary Computation.

[108]  Kay Chen Tan,et al.  Evolutionary Multi-objective Optimization in Uncertain Environments - Issues and Algorithms , 2009, Studies in Computational Intelligence.

[109]  Jürgen Branke,et al.  Multiswarms, exclusion, and anti-convergence in dynamic environments , 2006, IEEE Transactions on Evolutionary Computation.

[110]  Manuel López-Ibáñez,et al.  Ant colony optimization , 2010, GECCO '10.

[111]  K. Weicker,et al.  On evolution strategy optimization in dynamic environments , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[112]  Antonio Navarro,et al.  Adaptive classifier based on K-means clustering and dynamic programing , 1997, Electronic Imaging.

[113]  F. Glover,et al.  Handbook of Metaheuristics , 2019, International Series in Operations Research & Management Science.

[114]  Shengxiang Yang,et al.  A memetic algorithm with adaptive hill climbing strategy for dynamic optimization problems , 2009, Soft Comput..

[115]  John J. Grefenstette,et al.  Evolvability in dynamic fitness landscapes: a genetic algorithm approach , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[116]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[117]  Xiaodong Li,et al.  On performance metrics and particle swarm methods for dynamic multiobjective optimization problems , 2007, 2007 IEEE Congress on Evolutionary Computation.

[118]  F. Glover,et al.  Fundamentals of Scatter Search and Path Relinking , 2000 .

[119]  Kalyanmoy Deb,et al.  Parallelizing multi-objective evolutionary algorithms: cone separation , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[120]  Philippe Collard,et al.  From GAs to artificial immune systems: improving adaptation in time dependent optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[121]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[122]  Ben Paechter,et al.  Parallelization of population-based multi-objective meta-heuristics: An empirical study , 2006 .

[123]  Julio Ortega Lopera,et al.  Approaching Dynamic Multi-Objective Optimization Problems by Using Parallel Evolutionary Algorithms , 2010, Advances in Multi-Objective Nature Inspired Computing.

[124]  Enrique Alba,et al.  Parallel Metaheuristics: A New Class of Algorithms , 2005 .

[125]  Christoph F. Eick,et al.  Supporting Polyploidy in Genetic Algorithms Using Dominance Vectors , 1997, Evolutionary Programming.

[126]  Julio Ortega,et al.  A Pareto-based memetic algorithm for optimization of looped water distribution systems , 2010 .

[127]  David Wallace,et al.  Dynamic multi-objective optimization with evolutionary algorithms: a forward-looking approach , 2006, GECCO.

[128]  Julio Ortega Lopera,et al.  Performance Measures for Dynamic Multi-Objective Optimization , 2009, IWANN.