Cooperative relay selection policy using partially observable Markov decision process

In wireless mobile environment, the wireless channels are unreliable and change fast due to fading and users' movement. Since the wireless channel state is not directly observable at the receiver but is embedded in the received signal, this paper uses hidden Markov model (HMM) to model and predict the state of available relays, and considers relay selection as a stochastic process where the state of wireless channel varies randomly. The objective of the process is to select one relay among different alternatives in each time-slot according to their channel state information (CSI) with the goal of maximizing the long term information rate of the whole transmission period. We model the whole procedure of relay selection as a type of partially observable Markov decision processes (POMDPs), and the optimal policy of which relay will be selected at each time-slot can be acquired by solving POMDPs with dynamic programming-based hidden Markov model scheduling algorithms. Simulation results are presented to show the effectiveness of the proposed scheme.

[1]  Gregory W. Wornell,et al.  Cooperative diversity in wireless networks: Efficient protocols and outage behavior , 2004, IEEE Transactions on Information Theory.

[2]  Meng Yu,et al.  Is amplify-and-forward practically better than decode-and-forward or vice versa? , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[3]  Wei Yu,et al.  Joint Optimization of Relay Strategies and Resource Allocations in Cooperative Cellular Networks , 2006 .

[4]  A. Cassandra,et al.  Exact and approximate algorithms for partially observable markov decision processes , 1998 .

[5]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[6]  Jun Cai,et al.  Semi-Distributed User Relaying Algorithm for Amplify-and-Forward Wireless Relay Networks , 2008, IEEE Transactions on Wireless Communications.

[7]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[8]  Edward J. Sondik,et al.  The optimal control of par-tially observable Markov processes , 1971 .

[9]  E. J. Sondik,et al.  The Optimal Control of Partially Observable Markov Decision Processes. , 1971 .

[10]  K. J. Ray Liu,et al.  SPC12-5: Relay Selection in Multi-Node Cooperative Communications: When to Cooperate and Whom to Cooperate with? , 2006, IEEE Globecom 2006.

[11]  Richard D. Smallwood,et al.  Optimal Control of Partiality Observable Markov Processes over a Finite Horizon , 2012 .

[12]  M. Uysal,et al.  A novel relay selection method for decode-and-forward relaying , 2008, 2008 Canadian Conference on Electrical and Computer Engineering.

[13]  Aggelos Bletsas,et al.  A simple Cooperative diversity method based on network path selection , 2005, IEEE Journal on Selected Areas in Communications.

[14]  Vikram Krishnamurthy,et al.  A Value Iteration Algorithm for Partially Observed Markov Decision Process Multi-armed Bandits , 2004 .