Flying in the dark: controlling autonomous data ferries with partial observations

We seek to support communications in highly-partitioned mobile wireless networks via controllable data ferries. While existing ferry control techniques assume either stationary nodes or complete ferry observation of node locations, we address the more challenging scenario of highly mobile nodes and partial ferry observations. Using the tool of Partially Observable Markov Decision Processes (POMDP), we develop a comprehensive framework where we expand the solution space from predetermined trajectories to policies that can map ferry observations to navigation actions dynamically. Under this framework, we present an optimal and several efficient heuristic policies. We compare the proposed policies with predetermined control through analysis and simulations with respect to multiple node mobility parameters including speed, locality, activeness, and range of movement. The comparisons show a significant performance gain of up to twice the contact rate in cases of high uncertainty. In cases of low uncertainty, we give a sufficient condition under which predetermined control is optimal.

[1]  Timothy X. Brown,et al.  Towards autonomous data ferry route design through reinforcement learning , 2008, 2008 International Symposium on a World of Wireless, Mobile and Multimedia Networks.

[2]  Eytan Modiano,et al.  Mobile backbone networks --: construction and maintenance , 2006, MobiHoc '06.

[3]  Jie Wu,et al.  Logarithmic Store-Carry-Forward Routing in Mobile Ad Hoc Networks , 2007, IEEE Transactions on Parallel and Distributed Systems.

[4]  Benjamin Paul Jordan,et al.  Optimal Search for a Moving Target , 1995, Probability in the Engineering and Informational Sciences.

[5]  Joelle Pineau,et al.  Anytime Point-Based Approximations for Large POMDPs , 2006, J. Artif. Intell. Res..

[6]  Ellen W. Zegura,et al.  A message ferrying approach for data delivery in sparse mobile ad hoc networks , 2004, MobiHoc '04.

[7]  Qun Li,et al.  Sending messages to mobile users in disconnected ad-hoc wireless networks , 2000, MobiCom '00.

[8]  Milos Hauskrecht,et al.  Value-Function Approximations for Partially Observable Markov Decision Processes , 2000, J. Artif. Intell. Res..

[9]  Daniel Henkel,et al.  On Controlled Node Mobility in Delay-Tolerant Networks of Unmanned Aerial Vehicles , 2006 .

[10]  Ellen W. Zegura,et al.  Controlling the mobility of multiple data transport ferries in a delay-tolerant network , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[11]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[12]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs , 1978, Oper. Res..

[13]  E. J. Sondik,et al.  The Optimal Control of Partially Observable Markov Decision Processes. , 1971 .

[14]  Timothy X. Brown,et al.  Ad Hoc UAV Ground Network (AUGNet) , 2004 .