A Cooperative Retransmission Scheme in Wireless Networks with Imperfect Channel State Information

A transmitted packet that fails to reach its intended destination may be correctly received by neighbor nodes due to the broadcast nature of the wireless medium. In a cooperative retransmission scheme, these neighbor nodes, known as relays, can retransmit the failed packet on behalf of the original source node. The challenge is that multiple concurrent transmissions may lead to collision at the destination, and thus the problem is to decide which relay should help in retransmitting the failed packet so that the destination can successfully receive it. This paper proposes a decentralized partially observable Markov decision process (DEC-POMDP) model for selecting the relays to perform the cooperative retransmission. The proposed DEC-POMDP model does not require global channel state information (CSI). In addition, it is robust to noise in CSI measurements. Furthermore, the proposed DEC-POMDP scheme utilizes the gradient descent learning method to eliminate the need for a wireless channel model. We show that the proposed learning method based on the DEC-POMDP model can perform near optimally in the absence of a channel model and despite its implementation simplicity.

[1]  Mehul Motani,et al.  Buffer and channel adaptive modulation for transmission over fading channels , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[2]  Douglas Aberdeen,et al.  Policy-Gradient Algorithms for Partially Observable Markov Decision Processes , 2003 .

[3]  Chen-Khong Tham,et al.  A Low-Overhead Cooperative Retransmission Scheme for IR-UWB Networks , 2008, J. Electr. Comput. Eng..

[4]  M. Littman The Witness Algorithm: Solving Partially Observable Markov Decision Processes , 1994 .

[5]  K. J. Ray Liu,et al.  Near-optimal reinforcement learning framework for energy-aware sensor communications , 2005, IEEE Journal on Selected Areas in Communications.

[6]  Chen-Khong Tham,et al.  Coordinated Reinforcement Learning for Decentralized Optimal Control , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[7]  Kevin P. Murphy,et al.  A Survey of POMDP Solution Techniques , 2000 .

[8]  William S. Lovejoy,et al.  Computationally Feasible Bounds for Partially Observed Markov Decision Processes , 1991, Oper. Res..

[9]  Sagar Naik,et al.  A node-cooperative ARQ scheme for wireless ad hoc networks , 2005, IEEE Transactions on Vehicular Technology.

[10]  Andrew W. Moore,et al.  Distributed Value Functions , 1999, ICML.

[11]  Chen-Khong Tham,et al.  A Cooperative Retransmission Scheme for IR-UWB networks , 2008, 2008 IEEE International Conference on Ultra-Wideband.

[12]  Chen-Khong Tham,et al.  Markov Decision Process Frameworks for Cooperative Retransmission in Wireless Networks , 2009, 2009 IEEE Wireless Communications and Networking Conference.

[13]  Mehul Motani,et al.  Buffer and channel adaptive transmission over fading channels with imperfect channel state information , 2004, 2004 IEEE Wireless Communications and Networking Conference (IEEE Cat. No.04TH8733).

[14]  Hong Shen Wang,et al.  Finite-state Markov channel-a useful model for radio communication channels , 1995 .

[15]  Wai-Choong Wong,et al.  Performance analysis of a cooperative retransmission scheme using Markov models , 2007, 2007 6th International Conference on Information, Communications & Signal Processing.

[16]  G. Murphy,et al.  A survey of techniques for the analysis of sampled-data control systems , 1957 .