An adaptive opportunistic routing scheme for wireless ad-hoc networks

In this paper, an adaptive opportunistic routing scheme for multi-hop wireless ad-hoc networks is proposed. The proposed scheme utilizes a reinforcement learning framework to achieve the optimal performance even in the absence of reliable knowledge about channel statistics and network model. This scheme is shown to be optimal with respect to an expected average per packet cost criterion. The proposed routing scheme jointly addresses the issues of learning and routing in an opportunistic context, where the network structure is characterized by the transmission success probabilities. In particular, this learning framework leads to a stochastic routing scheme which optimally “explores” and “exploits” the opportunities in the network.

[1]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[2]  I Chih-Lin,et al.  Wireless Communications and Networks , 2004 .

[3]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[4]  Piet Van Mieghem,et al.  Responsible Editor: A. Kshemkalyani , 2006 .

[5]  Michele Zorzi,et al.  Geographic Random Forwarding (GeRaF) for Ad Hoc and Sensor Networks: Multihop Performance , 2003, IEEE Trans. Mob. Comput..

[6]  Javier A. Barria,et al.  A reinforcement learning ticket-based probing path discovery scheme for MANETs , 2004, Ad Hoc Networks.

[7]  Hideki Satoh,et al.  A Nonlinear Approach to Robust Routing Based on Reinforcement Learning with State Space Compression and Adaptive Basis Construction , 2008, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[8]  John S. Baras,et al.  A Probabilistic Emergent Routing Algorithm for Mobile Ad Hoc Networks , 2003 .

[9]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[10]  Robert Tappan Morris,et al.  Architecture and evaluation of an unplanned 802.11b mesh network , 2005, MobiCom '05.

[11]  Demosthenis Teneketzis,et al.  Stochastic routing in ad-hoc networks , 2006, IEEE Transactions on Automatic Control.

[12]  Dina Katabi,et al.  Zigzag decoding: combating hidden terminals in wireless networks , 2008, SIGCOMM '08.

[13]  Peter Larsson.  Selection diversity forwarding in a multihop packet radio network with fading channel and capture , 2001, MOCO.

[14]  Michael J. Neely,et al.  Optimal Backpressure Routing for Wireless Networks with Multi-Receiver Diversity , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[15]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[16]  Anatolij Zubow,et al.  Cooperative Opportunistic Routing Using Transmit Diversity in Wireless Mesh Networks , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[17]  Dit-Yan Yeung,et al.  Predictive Q-Routing: A Memory-based Reinforcement Learning Approach to Adaptive Traffic Control , 1995, NIPS.

[18]  T. Javidi,et al.  Towards Throughput and Delay Optimal Routing for Wireless Ad-Hoc Networks , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[19]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[20]  John N. Tsitsiklis,et al.  Asynchronous stochastic approximation and Q-learning , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[21]  William Stallings,et al.  Wireless Communications & Networks (2nd Edition) , 2004 .

[22]  S. Resnick.  A Probability Path , 1999 .

[23]  Peter Auer,et al.  Logarithmic Online Regret Bounds for Undiscounted Reinforcement Learning , 2006, NIPS.

[24]  Robert Tappan Morris,et al.  ExOR: opportunistic multi-hop routing for wireless networks , 2005, SIGCOMM '05.

[25]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[26]  Demosthenis Teneketzis,et al.  Stochastic routing in ad hoc wireless networks , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[27]  Eytan Modiano,et al.  Dynamic power allocation and routing for time-varying wireless networks , 2005 .

[28]  Elizabeth M. Belding-Royer,et al.  A review of current routing protocols for ad hoc mobile wireless networks , 1999, IEEE Wirel. Commun..

[29]  Samir Ranjan Das,et al.  Exploiting path diversity in the link layer in wireless ad hoc networks , 2005, Sixth IEEE International Symposium on a World of Wireless Mobile and Multimedia Networks.

[30]  Shailesh Kumar and Risto Miikkulainen.  Dual Reinforcement Q-Routing: An On-Line Adaptive Routing Algorithm , 1997 .

[31]  Sachin Katti,et al.  Trading structure for randomness in wireless opportunistic routing , 2007, SIGCOMM 2007.

[32]  G. van Dooren,et al.  Introduction to radio propagation for fixed and mobile communications , 1999, IEEE Antennas and Propagation Magazine.

[33]  Stuart J. Russell,et al.  Reinforcement Learning with Hierarchies of Machines , 1997, NIPS.