Cooperative retransmissions using Markov decision process with reinforcement learning

In cooperative retransmissions, nodes with better channel qualities help other nodes in retransmitting a failed packet to its intended destination. In this paper, we propose a cooperative retransmission scheme where each node makes local decision to cooperate or not to cooperate at what transmission power using a Markov decision process with reinforcement learning. With the reinforcement learning, the proposed scheme avoids solving an Markov decision process with a large number of states. Through simulations, we show that the proposed scheme is robust to collisions, is scalable with regard to the network size, and can provide significant cooperative diversity.

[1]  Chen-Khong Tham,et al.  Markov Decision Process Frameworks for Cooperative Retransmission in Wireless Networks , 2009, 2009 IEEE Wireless Communications and Networking Conference.

[2]  Zhongmin Shen,et al.  Projectively flat Finsler metrics of constant flag curvature , 2002 .

[3]  Sagar Naik,et al.  A node-cooperative ARQ scheme for wireless ad hoc networks , 2005, IEEE Transactions on Vehicular Technology.

[4]  Aytac Azgin,et al.  Cooperative MAC and routing protocols for wireless ad hoc networks , 2005, GLOBECOM '05. IEEE Global Telecommunications Conference, 2005..

[5]  Victor R. Lesser,et al.  Minimizing communication cost in a distributed Bayesian network using a decentralized MDP , 2003, AAMAS '03.

[6]  Matthew C. Valenti,et al.  Practical relay networks: a generalization of hybrid-ARQ , 2005 .

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Sathya Narayanan,et al.  CoopMAC: A Cooperative MAC for Wireless LANs , 2007, IEEE Journal on Selected Areas in Communications.

[9]  M. Fu,et al.  A distributed algorithm for solving a class of multi-agent Markov decision problems , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[10]  Chen-Khong Tham,et al.  A Cooperative Retransmission Scheme in Wireless Networks with Imperfect Channel State Information , 2009, 2009 IEEE Wireless Communications and Networking Conference.

[11]  Andrew W. Moore,et al.  Distributed Value Functions , 1999, ICML.

[12]  Hong Shen Wang,et al.  Finite-state Markov channel-a useful model for radio communication channels , 1995 .