An analytic modelling approach for network routing algorithms that use "ant-like" mobile agents

In this paper, we introduce an analytic modelling approach to the study of a novel class of adaptive network routing algorithm, which is inspired by the emergent problem-solving behaviours observed in biological ant colonies. This class of algorithm utilizes "ant-like" agents which traverse the network and collectively construct routing policies. Previous studies have focused exclusively on simulation experiments, which indicate that such algorithms perform well in response to real-time changes in traffic demands and network conditions. The analytic model presented in this paper permits useful insights into certain fundamental aspects of ant-based algorithms, which have not been discussed in previous ant-based routing literature. In particular, the work presented in this paper motivates our proposal of a number of modifications to the basic design of ant-based routing algorithms, which result in improved performance with respect to equilibrium performance measures.

[1]  Robert G. Gallager,et al.  A Minimum Delay Routing Algorithm Using Distributed Computation , 1977, IEEE Trans. Commun..

[2]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[3]  Aurel A. Lazar,et al.  On the existence of equilibria in noncooperative optimal flow control , 1995, JACM.

[4]  Dimitri P. Bertsekas,et al.  Dynamic behavior of shortest path routing algorithms for communication networks , 1982 .

[5]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[6]  Leonard Kleinrock,et al.  Theory, Volume 1, Queueing Systems , 1975 .

[7]  Alain Haurie,et al.  On the relationship between Nash - Cournot and Wardrop equilibria , 1983, Networks.

[8]  V. Borkar.  Asynchronous Stochastic Approximations , 1998 .

[9]  Stella C. Dafermos,et al.  Traffic assignment problem for a general network , 1969 .

[10]  J. G. Wardrop,et al.  Some Theoretical Aspects of Road Traffic Research , 1952 .

[11]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[12]  Luca Maria Gambardella,et al.  Ant Algorithms for Discrete Optimization , 1999, Artificial Life.

[13]  Dit-Yan Yeung,et al.  Predictive Q-Routing: A Memory-based Reinforcement Learning Approach to Adaptive Traffic Control , 1995, NIPS.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Vol. II , 1976 .

[16]  John N. Tsitsiklis,et al.  Actor-Critic Algorithms , 1999, NIPS.

[17]  Léon J. M. Rothkrantz,et al.  Ant-Based Load Balancing in Telecommunications Networks , 1996, Adapt. Behav..

[18]  G. Theraulaz,et al.  Inspiration for optimization from social insect behaviour , 2000, Nature.

[19]  Vivek S. Borkar,et al.  Dynamic Cesaro-Wardrop equilibration in networks , 2003, IEEE Trans. Autom. Control..

[20]  Dimitri P. Bertsekas,et al.  Data Networks , 1986 .

[21]  Martin Heusse,et al.  Adaptive Agent-Driven Routing and Load Balancing in Communication Networks , 1998, Adv. Complex Syst..

[22]  Mario Gerla,et al.  Optimal Routing in a Packet-Switched Computer Network , 1974, IEEE Transactions on Computers.

[23]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[24]  Marco Dorigo,et al.  From Natural to Artificial Swarm Intelligence , 1999 .

[25]  J. Deneubourg,et al.  The self-organizing exploratory pattern of the argentine ant , 1990, Journal of Insect Behavior.

[26]  Ariel Orda,et al.  Competitive routing in multi-user communication networks , 1993, IEEE INFOCOM '93 The Conference on Computer Communications, Proceedings.

[27]  Hisao Kameda,et al.  Mixed equilibrium (ME) for multiclass routing games , 2002, IEEE Trans. Autom. Control..

[28]  Marco Dorigo,et al.  AntNet: Distributed Stigmergetic Control for Communications Networks , 1998, J. Artif. Intell. Res..

[29]  Guy Theraulaz,et al.  Routing in Telecommunications Networks with Ant-Like Agents , 1999, IATA.

[30]  Donald F. Towsley,et al.  Distributed routing with on-line marginal delay estimation , 1990, IEEE Trans. Commun..

[31]  Mark M. Millonas,et al.  Swarms, Phase Transitions, and Collective Intelligence , 1993, adap-org/9306002.

[32]  Ariel Orda,et al.  Competitive routing in multiuser communication networks , 1993, TNET.

[33]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[34]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[35]  J. Wardrop.  ROAD PAPER. SOME THEORETICAL ASPECTS OF ROAD TRAFFIC RESEARCH. , 1952 .

[36]  Kwang Mong Sim,et al.  Ant colony optimization for routing and load-balancing: survey and new directions , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[37]  Dimitri P. Bertsekas,et al.  Second Derivative Algorithms for Minimum Delay Distributed Routing in Networks , 1984, IEEE Trans. Commun..

[38]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[39]  Vijay R. Konda,et al.  OnActor-Critic Algorithms , 2003, SIAM J. Control. Optim..

[40]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .