Decentralized Stochastic Control of Delay Tolerant Networks

We study in this paper optimal stochastic control issues in delay tolerant networks. We first derive the structure of optimal 2-hop forwarding policies. In order to be implemented, such policies require the knowledge of some system parameters such as the number of mobiles or the rate of contacts between mobiles, but these could be unknown at system design time or may change over time. To address this problem, we design adaptive policies combining estimation and control that achieve optimal performance in spite of the lack of information. We then study interactions that may occur in the presence of several competing classes of mobiles and formulate this as a cost-coupled stochastic game. We show that this game has a unique Nash equilibrium such that each class adopts the optimal forwarding policy determined for the single class problem.

[1]  Rachid El Azouzi,et al.  Constrained Stochastic Games in Wireless Networks , 2007, IEEE GLOBECOM 2007 - IEEE Global Telecommunications Conference.

[2]  Ellen W. Zegura,et al.  Controlling the mobility of multiple data transport ferries in a delay-tolerant network , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[3]  Mirco Musolesi,et al.  Controlled Epidemic-style Dissemination Middleware for Mobile Ad Hoc Networks , 2006, 2006 Third Annual International Conference on Mobile and Ubiquitous Systems: Networking & Services.

[4]  E. Altman.  Constrained Markov Decision Processes , 1999 .

[5]  Jean-Yves Le Boudec,et al.  Performance Analysis of Self Limiting Epidemic Forwarding , 2006 .

[6]  Ger Koole,et al.  Message delay in MANET , 2005, SIGMETRICS '05.

[7]  Vinton G. Cerf,et al.  Delay-tolerant networking: an approach to interplanetary Internet , 2003, IEEE Commun. Mag..

[8]  ChaintreauAugustin,et al.  Impact of Human Mobility on Opportunistic Forwarding Algorithms , 2007 .

[9]  J. L. Maryak.  Some guidelines for using iterate averaging in stochastic approximation , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[10]  Ger Koole,et al.  The message delay in mobile ad hoc networks , 2005, Perform. Evaluation.

[11]  Eitan Altman,et al.  Constrained cost-coupled stochastic games with independent state processes , 2007, Oper. Res. Lett..

[12]  Eitan Altman,et al.  Optimal monotone forwarding policies in delay tolerant mobile ad-hoc networks , 2008, Valuetools 2008.

[13]  Moshe Shaked,et al.  Stochastic orders and their applications , 1994 .

[14]  Ellen W. Zegura,et al.  Message ferry route design for sparse ad hoc networks with mobile nodes , 2006, MobiHoc '06.

[15]  Marco Conti,et al.  Opportunistic networking: data forwarding in disconnected mobile ad hoc networks , 2006, IEEE Communications Magazine.

[16]  Thrasyvoulos Spyropoulos,et al.  Optimal Buffer Management Policies for Delay Tolerant Networks , 2008, 2008 5th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks.

[17]  T. Spyropoulos,et al.  Efficient Routing in Intermittently Connected Mobile Networks: The Multiple-Copy Case , 2008, IEEE/ACM Transactions on Networking.

[18]  Christophe Diot,et al.  Impact of Human Mobility on Opportunistic Forwarding Algorithms , 2007, IEEE Transactions on Mobile Computing.

[19]  Giovanni Neglia,et al.  Optimal delay-power tradeoff in sparse delay tolerant networks: a preliminary study , 2006, CHANTS '06.

[20]  Amin Vahdat,et al.  Epidemic Routing for Partially-Connected Ad Hoc Networks , 2009 .

[21]  K. Psounis,et al.  Efficient Routing in Intermittently Connected Mobile Networks: The Single-Copy Case , 2008, IEEE/ACM Transactions on Networking.

[22]  Jean-Yves Le Boudec,et al.  Power Law and Exponential Decay of Intercontact Times between Mobile Devices , 2007, IEEE Transactions on Mobile Computing.

[23]  Donald F. Towsley,et al.  Performance Modeling of Epidemic Routing , 2006, Networking.

[24]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[25]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .