Convergence results for ant routing algorithms via stochastic approximation

In this article, we provide convergence results for an Ant-based Routing Algorithm (ARA) for wireline, packet-switched communication networks, that are acyclic. Such algorithms are inspired by the foraging behavior of ants in nature. We consider an ARA algorithm proposed earlier by Bean and Costa [2005]. The algorithm has the virtues of being adaptive and distributed, and can provide a multipath routing solution. We consider a scenario where there are multiple incoming data traffic streams that are to be routed to their respective destinations via the network. Ant packets, which are nothing but probe packets, are introduced to estimate the path delays in the network. The node routing tables, which consist of routing probabilities for the outgoing links, are updated based on these delay estimates. In contrast to the available analytical studies in the literature, the link delays in our model are stochastic, time-varying, and dependent on the link traffic. The evolution of the delay estimates and the routing probabilities are described by a set of stochastic iterative equations. In doing so, we take into account the distributed and asynchronous nature of the algorithm operation. Using methods from the theory of stochastic approximations, we show that the evolution of the delay estimates can be closely tracked by a deterministic ODE (Ordinary Differential Equation) system, when the step size of the delay estimation scheme is small. We study the equilibrium behavior of the ODE system in order to obtain the equilibrium behavior of the routing algorithm. We also explore properties of the equilibrium routing probabilities, and provide illustrative simulation results.

[1]  John S. Baras,et al.  Convergence results for ant routing algorithms via stochastic approximation and optimization , 2010, 2007 46th IEEE Conference on Decision and Control.

[2]  Vivek S. Borkar,et al.  A novel ACO algorithm for optimization via reinforcement and initial bias , 2009, Swarm Intelligence.

[3]  M. Thathachar,et al.  Networks of Learning Automata: Techniques for Online Stochastic Optimization , 2003 .

[4]  W. Gutjahr.  On the Finite-Time Dynamics of Ant Colony Optimization , 2006 .

[5]  M. Hirsch.  Systems of Differential Equations that are Competitive or Cooperative II: Convergence Almost Everywhere , 1985 .

[6]  Ke Li,et al.  Modeling, analysis and simulation of ant-based network routing protocols , 2010, Swarm Intelligence.

[7]  Luca Maria Gambardella,et al.  Principles and applications of swarm intelligence for adaptive routing in telecommunications networks , 2010, Swarm Intelligence.

[8]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[9]  V. Borkar.  Stochastic Approximation: A Dynamical Systems Viewpoint , 2008 .

[10]  Armand M. Makowski,et al.  The Binary Bridge Selection Problem: Stochastic Approximations and the Convergence of a Learning Algorithm , 2008, ANTS Conference.

[11]  Christian Blum,et al.  Ant colony optimization for multicasting in static wireless ad-hoc networks , 2009, Swarm Intelligence.

[12]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[13]  Isi Mitrani,et al.  Probabilistic Modelling , 1998 .

[14]  Devika Subramanian,et al.  Ants and Reinforcement Learning: A Case Study in Routing in Dynamic Networks , 1997, IJCAI.

[15]  Muddassar Farooq,et al.  Swarm intelligence based routing protocol for wireless sensor networks: Survey and future directions , 2011, Inf. Sci..

[16]  Martin Roth,et al.  The Markovian Termite: A Soft Routing Framework , 2007, 2007 IEEE Swarm Intelligence Symposium.

[17]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[18]  Luca Maria Gambardella,et al.  AntHocNet: an adaptive nature-inspired algorithm for routing in mobile ad hoc networks , 2005, Eur. Trans. Telecommun..

[19]  Eran Gabber,et al.  Trail blazer: a routing algorithm inspired by ants , 2004, Proceedings of the 12th IEEE International Conference on Network Protocols, 2004. ICNP 2004..

[20]  Nigel Bean,et al.  An analytic modelling approach for network routing algorithms that use "ant-like" mobile agents , 2005, Comput. Networks.

[21]  M. A. L. Thathachar,et al.  Networks of Learning Automata , 2004 .

[22]  W. Gutjahr.  A GENERALIZED CONVERGENCE RESULT FOR THE GRAPH-BASED ANT SYSTEM METAHEURISTIC , 2003, Probability in the Engineering and Informational Sciences.

[23]  Vivek S. Borkar,et al.  Dynamic Cesaro-Wardrop equilibration in networks , 2003, IEEE Trans. Autom. Control..

[24]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[25]  Marco Dorigo,et al.  AntNet: Distributed Stigmergetic Control for Communications Networks , 1998, J. Artif. Intell. Res..

[26]  Dimitri P. Bertsekas,et al.  Data networks (2nd ed.) , 1992 .

[27]  John S. Baras,et al.  A Probabilistic Emergent Routing Algorithm for Mobile Ad Hoc Networks , 2003 .

[28]  John S. Baras,et al.  Convergence results for ant routing algorithms via stochastic approximation and optimization , 2007, CDC.

[29]  J. Deneubourg,et al.  The self-organizing exploratory pattern of the argentine ant , 1990, Journal of Insect Behavior.