Parallel Hybrid Optimization Methods For Permutation Based Problems

Solving efficiently large benchmarks of NP-hard permutation-based problems requires the development of hybrid methods combining different classes of optimization methods. Indeed, it is now acknowledged that such methods perform better than traditional optimization methods when used separately. The key challenge is how to find connections between the divergent search strategies used in each class of methods in order to build efficient hybridization strategies. Genetic algorithms (GAs) are very popular population-based metaheuristics based on stochastic evolutionary operators. The hybridization of GAs with tree-based exact methods such as Branch-and-Bound is a promising research trend. B&B algorithms are based on an implicit enumeration of the solution space represented as a tree. Our hybridization approach consists in providing a common solution and search space coding and associated search operators enabling an efficient cooperation between the two methods. The tree-based representation of the solution space is traditionally used in B&B algorithms to enumerate the solutions of the problem at hand. In this thesis, this special representation is adapted to metaheuristics. The encoding of permutations as natural numbers, which refer to their lexicographic enumeration in the tree, is proposed as a new way to represent the solution space of permutation problems in metaheuristics. This encoding approach is based on the mathematical properties of permutations (Lehmer codes, inversion tables, etc.). Mapping functions between the two representations (permutations and numbers) and special search operators adapted to the encoding are defined for general permutation problems, with respect to the theory of representation. This common representation allows the design of efficient cooperation strategies between GAs and B&B algorithms. In this thesis, two hybridization schemes combining GAs with B&B based on this common representation are proposed. The two hybridization approaches HGABB/HAGABB (Hybrid Adaptive GA-B&B) and COBBIGA (cooperative B&B interval-based GA), have been validated on standard benchmarks of one of the hardest permutation-based problems, the three dimensional quadratic assignment problem (Q3AP). In order to solve large benchmarks of permutationbased problems, a parallelization for computational grids is also proposed for the two hybrid schemes. This parallelization is based on space decomposition techniques (the decomposition by intervals) used in parallel B&B algorithms. From the implementation point of view, in order to facilitate further design and implementation of hybrid methods combining metaheuristics with tree-based exact methods, a hybridization C++ framework integrated to the framework for metaheuristics ParadisEO is developed. The new framework is used to conduct extensive experiments over the computational grid Grid'5000.

[1]  Jean-Louis Bouquard,et al.  Genetic Branch-and-Bound or Exact Genetic Algorithm? , 2007, Artificial Evolution.

[2]  P. Miliotis,et al.  An algorithm for the planar three-index assignment problem , 1994 .

[3]  David E. Goldberg,et al.  Alleles, loci and the traveling salesman problem , 1985 .

[4]  D. Atkin OR scheduling algorithms. , 2000, Anesthesiology.

[5]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[6]  Bernard Gendron,et al.  Parallel Branch-and-Branch Algorithms: Survey and Synthesis , 1994, Oper. Res..

[7]  Enrique Alba,et al.  Influence of the Migration Policy in Parallel Distributed GAs with Structured and Panmictic Populations , 2000, Applied Intelligence.

[8]  Günther R. Raidl,et al.  Cooperating Memetic and Branch-and-Cut Algorithms for Solving the Multidimensional Knapsack Problem , 2005 .

[9]  A. N. Elshafei,et al.  Hospital Layout as a Quadratic Assignment Problem , 1977 .

[10]  El-Ghazali Talbi,et al.  A GPU-based iterated tabu search for solving the quadratic 3-dimensional assignment problem , 2010, ACS/IEEE International Conference on Computer Systems and Applications - AICCSA 2010.

[11]  El-Ghazali Talbi,et al.  ParadisEO: A Framework for the Reusable Design of Parallel and Distributed Metaheuristics , 2004, J. Heuristics.

[12]  Mauro Dell'Amico,et al.  Assignment Problems , 1998, IFIP Congress: Fundamentals - Foundations of Computer Science.

[13]  L. Darrell Whitley,et al.  Cellular Genetic Algorithms , 1993, ICGA.

[14]  Günther R. Raidl,et al.  A Unified View on Hybrid Metaheuristics , 2006, Hybrid Metaheuristics.

[15]  Eduardo L. Pasiliao Local Neighborhoods for the Multidimensional Assignment Problem , 2010 .

[16]  S. M. Johnson,et al.  Optimal two- and three-stage production schedules with setup times included , 1954 .

[17]  Rajkumar Buyya,et al.  Grids and Grid technologies for wide‐area distributed computing , 2002, Softw. Pract. Exp..

[18]  Catherine Roucairol,et al.  Bob++: Framework for Solving Optimization Problems with Branch-and-Bound methods , 2006, 2006 15th IEEE International Conference on High Performance Distributed Computing.

[19]  Bum-Jin Kim,et al.  Investigation of methods for solving new classes of quadratic assignment problems (QAPs) , 2006 .

[20]  Vassilis Zissimopoulos,et al.  On the Hardness of the Quadratic Assignment Problem with Metaheuristics , 2002, J. Heuristics.

[21]  Bertrand Le Cun,et al.  A Parallel Exact Solver for the Three-Index Quadratic Assignment Problem , 2011, 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum.

[22]  Franz Rothlauf,et al.  Representations for genetic and evolutionary algorithms , 2002, Studies in Fuzziness and Soft Computing.

[23]  Catherine Roucairol,et al.  Difficulties of Exact Methods for Solving the Quadratic Assignment Problem , 1993, Quadratic Assignment and Related Problems.

[24]  Teofilo F. Gonzalez,et al.  P-Complete Approximation Problems , 1976, J. ACM.

[25]  Zbigniew Skolicki,et al.  The influence of migration sizes and intervals on island models , 2005, GECCO '05.

[26]  Brian W. Kernighan,et al.  An Effective Heuristic Algorithm for the Traveling-Salesman Problem , 1973, Oper. Res..

[27]  In Chan Choi,et al.  A hybrid genetic algorithm for train sequencing in the Korean railway , 2009 .

[28]  Zhi Ding,et al.  The quadratic three-dimensional assignment problem: Exact and approximate solution methods , 2008, Eur. J. Oper. Res..

[29]  Enrique Alba,et al.  Analyzing synchronous and asynchronous parallel distributed genetic algorithms , 2001, Future Gener. Comput. Syst..

[30]  Al Costanzo Branch-and-Bound with Peer-to-Peer for Large-Scale Grids , 2007 .

[31]  Enrique Alba,et al.  Parallel evolutionary algorithms can achieve super-linear performance , 2002, Inf. Process. Lett..

[32]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[33]  S. Ronald,et al.  More distance functions for order-based encodings , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[34]  T. Koopmans,et al.  Assignment Problems and the Location of Economic Activities , 1957 .

[35]  Zvi Drezner,et al.  Extensive experiments with hybrid genetic algorithms for the solution of the quadratic assignment problem , 2008, Comput. Oper. Res..

[36]  Thomas Stützle,et al.  A simple and effective iterated greedy algorithm for the permutation flowshop scheduling problem , 2007, Eur. J. Oper. Res..

[37]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[38]  Erick Cantú-Paz,et al.  Modeling Idealized Bounding Cases of Parallel Genetic Algorithms , 1996 .

[39]  Christian Bierwirth,et al.  On Permutation Representations for Scheduling Problems , 1996, PPSN.

[40]  K. Miettinen,et al.  Quasi-random initial population for genetic algorithms , 2004 .

[41]  S. Pesko DIFFERENTIAL EVOLUTION FOR SMALL TSPs WITH CONSTRAINTS , 2006 .

[42]  Enrique Alba,et al.  Parallelism and evolutionary algorithms , 2002, IEEE Trans. Evol. Comput..

[43]  David Thomas,et al.  The Art in Computer Programming , 2001 .

[44]  Panos M. Pardalos,et al.  Quadratic Assignment Problem , 1997, Encyclopedia of Optimization.

[45]  Zhi Ding,et al.  Optimal symbol mapping diversity for multiple packet transmissions , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[46]  Franz Rendl,et al.  QAPLIB – A Quadratic Assignment Problem Library , 1997, J. Glob. Optim..

[47]  El-Ghazali Talbi,et al.  A Taxonomy of Hybrid Metaheuristics , 2002, J. Heuristics.

[48]  Dacheng Yang,et al.  A hybrid ARQ scheme with constellation rearrangement and power adjustment , 2005, Proceedings. 2005 International Conference on Wireless Communications, Networking and Mobile Computing, 2005..

[49]  A. Frieze Complexity of a 3-dimensional assignment problem , 1983 .

[50]  Pascal Bouvry,et al.  Parallel Hybrid Genetic Algorithms for Solving Q3AP on Computational Grid , 2012, Int. J. Found. Comput. Sci..

[51]  Piotr Lipinski A Hybrid Evolutionary Algorithm to Quadratic Three-Dimensional Assignment Problem with Local Search for Many-Core Graphics Processors , 2010, IDEAL.

[52]  David E. Goldberg,et al.  Efficient Parallel Genetic Algorithms: Theory and Practice , 2000 .

[53]  Zdenek Konfrst,et al.  Parallel Genetic Algorithms: Advances, Computing Trends, Applications and Perspectives , 2004, IPDPS.

[54]  Panos M. Pardalos,et al.  GRASP With Path Relinking For The Three-Index Assignment Problem , 2000 .

[55]  Antonin Ponsich,et al.  Testing the Permutation Space Based Geometric Differential Evolution on the Job-Shop Scheduling Problem , 2010, PPSN.

[56]  Teodor Gabriel Crainic,et al.  PARALLEL BRANCH-AND-BOUND ALGORITHMS: SURVEY AND SYNTHESIS , 1993 .

[57]  El-Ghazali Talbi,et al.  A Grid-enabled Branch and Bound Algorithm for Solving Challenging Combinatorial Optimization Problems , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[58]  John H. Holland,et al.  Outline for a Logical Theory of Adaptive Systems , 1962, JACM.

[59]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[60]  Günther R. Raidi A unified view on hybrid metaheuristics , 2006 .

[61]  John W. Dickey,et al.  Campus building arrangement using topaz , 1972 .

[62]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[63]  Hamed Samarghandi,et al.  A Particle Swarm Optimization for the Single Row Facility Layout Problem , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[64]  John M. Wilson,et al.  Using a Hybrid Genetic-Algorithm/Branch and Bound Approach to Solve Feasibility and Optimization Integer Programming Problems , 2001, J. Heuristics.

[65]  Panos M. Pardalos,et al.  A Greedy Randomized Adaptive Search Procedure for the Quadratic Assignment Problem , 1993, Quadratic Assignment and Related Problems.

[66]  Ian T. Foster,et al.  The anatomy of the grid: enabling scalable virtual organizations , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[67]  Pascal Bouvry,et al.  A parallel hybrid genetic algorithm-simulated annealing for solving Q3AP on computational grid , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[68]  Alan Frieze,et al.  An Algorithm for Solving 3-Dimensional Assignment Problems with Application to Scheduling a Teaching Practice , 1981 .

[69]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[70]  Charles Fleurent,et al.  Genetic Hybrids for the Quadratic Assignment Problem , 1993, Quadratic Assignment and Related Problems.

[71]  Helena Ramalhinho Dias Lourenço,et al.  Iterated Local Search , 2001, Handbook of Metaheuristics.

[72]  Carlos Cotta,et al.  On the Hybridization of Memetic Algorithms With Branch-and-Bound Techniques , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[73]  D. J. Smith,et al.  A Study of Permutation Crossover Operators on the Traveling Salesman Problem , 1987, ICGA.

[74]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[75]  Monique Guignard-Spielberg,et al.  Exact solution of emerging quadratic assignment problems , 2010, Int. Trans. Oper. Res..

[76]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[77]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[78]  Kento Aida,et al.  Distributed computing with hierarchical master-worker paradigm for parallel branch and bound algorithm , 2003, CCGrid 2003. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003. Proceedings..

[79]  Didier El Baz,et al.  Solution of multidimensional knapsack problems via cooperation of dynamic programming and branch and bound , 2010 .

[80]  M. Kamel,et al.  A Taxonomy of Cooperative Search Algorithms , 2005, Hybrid Metaheuristics.

[81]  Nair Maria Maia de Abreu,et al.  A survey for the quadratic assignment problem , 2007, Eur. J. Oper. Res..

[82]  Rajkumar Buyya,et al.  A taxonomy and survey of grid resource management systems for distributed computing , 2002, Softw. Pract. Exp..

[83]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[84]  E. L. Lawler,et al.  Branch-and-Bound Methods: A Survey , 1966, Oper. Res..

[85]  Thomas Sttzle,et al.  Applying iterated local search to the permutation ow shop problem , 1998 .

[86]  Günther R. Raidl,et al.  Combining Metaheuristics and Exact Algorithms in Combinatorial Optimization: A Survey and Classification , 2005, IWINAC.

[87]  Thomas Stützle,et al.  Iterated local search for the quadratic assignment problem , 2006, Eur. J. Oper. Res..

[88]  R. Burkard,et al.  Computational investigations on 3-dimensional axial assignment problems , 1993 .

[89]  Pedro Larrañaga,et al.  Genetic Algorithms for the Travelling Salesman Problem: A Review of Representations and Operators , 1999, Artificial Intelligence Review.

[90]  Gilbert Laporte,et al.  Metaheuristics: A bibliography , 1996, Ann. Oper. Res..

[91]  Jaime Simão Sichman,et al.  Using A-Teams to optimize automatic insertion of electronic components , 2003, Adv. Eng. Informatics.

[92]  Shahryar Rahnamayan,et al.  A novel population initialization method for accelerating evolutionary algorithms , 2007, Comput. Math. Appl..

[93]  Víctor Robles,et al.  Voronoi-initializated island models for solving real-coded deceptive problems , 2008, GECCO '08.

[94]  Selmer M. Johnson Generation of permutations by adjacent transposition , 1963 .

[95]  Mario Vanhoucke,et al.  Meta-Heuristic resource constrained project scheduling: solution space restrictions and neighbourhood extensions , 2006 .

[96]  El-Ghazali Talbi,et al.  Hybridizing exact methods and metaheuristics: A taxonomy , 2009, Eur. J. Oper. Res..

[97]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[98]  Pascal Bouvry,et al.  A cooperative tree-based hybrid GA-B&B approach for solving challenging permutation-based problems. , 2011, GECCO '11.

[99]  Mohamed Ben-Daya,et al.  A tabu search approach for the flow shop scheduling problem , 1998, Eur. J. Oper. Res..

[100]  Phillipe D'Anfray,et al.  GRID'5000 une plate-forme d'expérimentation pour les systèmes distribués à large échelle , 2007 .

[101]  Erick Cantú-Paz,et al.  A Survey of Parallel Genetic Algorithms , 2000 .

[102]  Christian Blum,et al.  Metaheuristics in combinatorial optimization: Overview and conceptual comparison , 2003, CSUR.

[103]  Kalyanmoy Deb,et al.  Ordering Genetic Algorithms and Deception , 1992, PPSN.

[104]  Pascal Bouvry,et al.  P2P computing for large tree exploration-based exact optimisation , 2009, Int. J. Grid Util. Comput..

[105]  Hale F. Trotter,et al.  Algorithm 115: Perm , 1962, CACM.

[106]  Roberto Mantaci,et al.  A permutations representation that knows what "Eulerian" means , 2001, Discret. Math. Theor. Comput. Sci..

[107]  Ian T. Foster,et al.  Condor-G: A Computation Management Agent for Multi-Institutional Grids , 2004, Cluster Computing.

[108]  Pascal Bouvry,et al.  Interval-based initialization method for permutation-based problems , 2010, IEEE Congress on Evolutionary Computation.

[109]  Thomas Stützle,et al.  Local search algorithms for combinatorial problems: analysis, algorithms, and new applications , 1999 .