Metaheuristic algorithms and probabilistic behaviour: a comprehensive analysis of Ant Colony Optimization and its variants

The application of metaheuristic algorithms to combinatorial optimization problems is on the rise and is growing rapidly now than ever before. In this paper the historical context and the conducive environment that accelerated this particular trend of inspiring analogies or metaphors from various natural phenomena are analysed. We have implemented the Ant System Model and the other variants of ACO including the 3-Opt, Max–Min, Elitist and the Rank Based Systems as mentioned in their original works and we converse the missing pieces of Dorigo’s Ant System Model. Extensive analysis of the variants on Travelling Salesman Problem and Job Shop Scheduling Problem shows how much they really contribute towards obtaining better solutions. The stochastic nature of these algorithms has been preserved to the maximum extent to keep the implementations as generic as possible. We observe that stochastic implementations show greater resistance to changes in parameter values, still obtaining near optimal solutions. We report how Polynomial Turing Reduction helps us to solve Job Shop Scheduling Problem without making considerable changes in the implementation of Travelling Salesman Problem, which could be extended to solve other NP-Hard problems. We elaborate on the various parallelization options based on the constraints enforced by strong scaling (fixed size problem) and weak scaling (fixed time problem). Also we elaborate on how probabilistic behaviour helps us to strike a balance between intensification and diversification of the search space.

[1]  A. Fraser.  Simulation of Genetic Systems by Automatic Digital Computers VI. Epistasis , 1960 .

[2]  Seyed Mohammad Mirjalili,et al.  The Ant Lion Optimizer , 2015, Adv. Eng. Softw..

[3]  Lawrence Snyder,et al.  Type architectures, shared memory, and the corollary of modest potential , 1986 .

[4]  J. Deneubourg,et al.  Collective patterns and decision-making , 1989 .

[5]  Wen-Tsao Pan,et al.  A new Fruit Fly Optimization Algorithm: Taking the financial distress model as an example , 2012, Knowl. Based Syst..

[6]  Jing Wang,et al.  Swarm Intelligence in Cellular Robotic Systems , 1993 .

[7]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  Mohammad Kazem Sayadia,et al.  A discrete firefly metaheuristic with local search for makespan minimization in permutation flow shop scheduling problems , 2010 .

[10]  Walter J. Gutjahr,et al.  A Graph-based Ant System and its convergence , 2000, Future Gener. Comput. Syst..

[11]  Gilbert Laporte,et al.  Metaheuristics: A bibliography , 1996, Ann. Oper. Res..

[12]  Marco Dorigo,et al.  Distributed Optimization by Ant Colonies , 1992 .

[13]  B Rampriya,et al.  Unit commitment in deregulated power system using Lagrangian firefly algorithm , 2010, 2010 INTERNATIONAL CONFERENCE ON COMMUNICATION CONTROL AND COMPUTING TECHNOLOGIES.

[14]  Kenneth Sörensen,et al.  Metaheuristics - the metaphor exposed , 2015, Int. Trans. Oper. Res..

[15]  Constantin Oprean,et al.  Elitist ant system for route allocation problem , 2008 .

[16]  Theofanis Apostolopoulos,et al.  Application of the Firefly Algorithm for Solving the Economic Emissions Load Dispatch Problem , 2011 .

[17]  Slawomir Zak,et al.  Firefly Algorithm for Continuous Constrained Optimization Tasks , 2009, ICCCI.

[18]  Li Cheng,et al.  A New Metaheuristic Bat-Inspired Algorithm , 2010 .

[19]  Deepak K. Gupta,et al.  Recursive Ant Colony Optimization for estimation of parameters of a function , 2012, 2012 1st International Conference on Recent Advances in Information Technology (RAIT).

[20]  G. Amdhal,et al.  Validity of the single processor approach to achieving large scale computing capabilities , 1967, AFIPS '67 (Spring).

[21]  Daniel J Poole,et al.  2014 IEEE Congress on Evolutionary Computation (CEC), Beijing, China , 2014 .

[22]  Fred W. Glover,et al.  Tabu Search - Part I , 1989, INFORMS J. Comput..

[23]  B. Bullnheimer,et al.  A NEW RANK BASED VERSION OF THE ANT SYSTEM: A COMPUTATIONAL STUDY , 1997 .

[24]  Harish Sharma,et al.  Artificial bee colony algorithm: a survey , 2013, Int. J. Adv. Intell. Paradigms.

[25]  Debasish Ghose,et al.  Detection of multiple source locations using a glowworm metaphor with applications to collective robotics , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[26]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[27]  Xin-She Yang,et al.  A New Metaheuristic Bat-Inspired Algorithm , 2010, NICSO.

[28]  Corso Elvezia,et al.  Ant colonies for the traveling salesman problem , 1997 .

[29]  Omid Bozorg Haddad,et al.  Honey-Bees Mating Optimization (HBMO) Algorithm: A New Heuristic Approach for Water Resources Optimization , 2006 .

[30]  Thomas Stützle,et al.  MAX-MIN Ant System , 2000, Future Gener. Comput. Syst..

[31]  Craig A. Tovey,et al.  On Honey Bees and Dynamic Server Allocation in Internet Hosting Centers , 2004, Adapt. Behav..

[32]  Satish Talreja.  A Heuristic Proposal in the Dimension of Ant Colony Optimization , 2013 .

[33]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1998 .

[34]  Dervis Karaboga,et al.  AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION , 2005 .

[35]  Günter Schmidt,et al.  Scheduling with limited machine availability , 2000, Eur. J. Oper. Res..

[36]  Xin-She Yang,et al.  Nature-Inspired Metaheuristic Algorithms , 2008 .

[37]  M. R. Arshad,et al.  A novel Fly Optimization Algorithm for swarming application , 2010, 2010 IEEE Conference on Robotics, Automation and Mechatronics.

[38]  Leandro dos Santos Coelho,et al.  A chaotic firefly algorithm applied to reliability-redundancy optimization , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[39]  M. Sayadi,et al.  A discrete firefly meta-heuristic with local search for makespan minimization in permutation flow shop scheduling problems , 2010 .

[40]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[41]  Ulf Grenander,et al.  A stochastic nonlinear model for coordinated bird flocks , 1990 .

[42]  J. Doll,et al.  Quantum annealing: A new method for minimizing multidimensional functions , 1994, chem-ph/9404003.

[43]  M. Dorigo,et al.  Ant System: An Autocatalytic Optimizing Process , 1991 .

[44]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[45]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[46]  David E. Goldberg,et al.  Genetic algorithms and Machine Learning , 1988, Machine Learning.

[47]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[48]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[49]  M Dorigo,et al.  Ant colonies for the travelling salesman problem. , 1997, Bio Systems.

[50]  David E. Goldberg,et al.  The Theory of Virtual Alphabets , 1990, PPSN.

[51]  F. Glover.  HEURISTICS FOR INTEGER PROGRAMMING USING SURROGATE CONSTRAINTS , 1977 .

[52]  John L. Gustafson,et al.  Reevaluating Amdahl's law , 1988, CACM.

[53]  张军,et al.  Orthogonal Methods Based Ant Colony Search for Solving Continuous Optimization Problems , 2008 .

[54]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[55]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[56]  Darrell Whitley,et al.  A genetic algorithm tutorial , 1994, Statistics and Computing.

[57]  Manijeh Keshtgari,et al.  Termite colony optimization: A novel approach for optimizing continuous problems , 2010, 2010 18th Iranian Conference on Electrical Engineering.

[58]  J. Deneubourg,et al.  Probabilistic behaviour in ants: A strategy of errors? , 1983 .

[59]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[60]  Hong Wang,et al.  Bacterial Colony Optimization , 2012 .