A novel cooperative communication protocol for QoS provisioning in wireless sensor networks

Cooperative communications have been demonstrated to be effective in combating the multiple fading effects in wireless networks, and improving the network performance in terms of adaptivity, reliability, data throughput and network life time. In this paper, we investigate the use of cooperative communications for quality of service (QoS) provisioning in resource-constrained wireless sensor networks, and propose MRL-CC, a Multi-agent Reinforcement Learning based multi-hop mesh Cooperative Communication mechanism for wireless sensor networks. In order to disseminate data reliably in MRL-CC, a multi-hop mesh cooperative structure is first constructed. Then a cooperative mechanism with cooperative partner assignments, and coding and transmission schemes is implemented using a multi-agent reinforcement learning algorithm. We compare the network performance of MRL-CC with MMCC [1], a Multi-hop Mesh structure based Cooperative Communication scheme, and investigate the impacts of network traffic load, interference and sensor node's mobility on the network performance. Simulation results show that MRL-CC performs well in terms of a number of QoS metrics, and fits well in large-scale networks and highly dynamic environments.

[1]  Zhenzhen Liu,et al.  RL-MAC: A QoS-Aware Reinforcement Learning based MAC Protocol for Wireless Sensor Networks , 2006, 2006 IEEE International Conference on Networking, Sensing and Control.

[2]  Mahmood Fathy,et al.  Prediction Based QoS Routing in MANETs , 2006, ICDCN.

[3]  Aria Nosratinia,et al.  Cooperation diversity through coding , 2002, Proceedings IEEE International Symposium on Information Theory,.

[4]  K. J. Ray Liu,et al.  When Does Cooperation Have Better Performance in Sensor Networks? , 2006, 2006 3rd Annual IEEE Communications Society on Sensor and Ad Hoc Communications and Networks.

[5]  Dongmei Zhang,et al.  A Q-Learning-based Decision Making Scheme for Application Reconfiguration in Sensor Networks , 2007, 2007 11th International Conference on Computer Supported Cooperative Work in Design.

[6]  Gregory W. Wornell,et al.  Cooperative diversity in wireless networks: Efficient protocols and outage behavior , 2004, IEEE Transactions on Information Theory.

[7]  Vincent W. S. Wong,et al.  Cooperative Protocols Design for Wireless Ad-Hoc Networks with Multi-hop Routing , 2008, QShine '08.

[8]  Athanassios Boulis,et al.  From Simulation to Real Deployments in WSN and Back , 2007, 2007 IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks.

[9]  Elza Erkip,et al.  User cooperation diversity. Part II. Implementation aspects and performance analysis , 2003, IEEE Trans. Commun..

[10]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[11]  K. J. Ray Liu,et al.  Cooperative communications with relay-selection: when to cooperate and whom to cooperate with? , 2008, IEEE Transactions on Wireless Communications.

[12]  H. T. Mouftah,et al.  QoS routing for wireless ad hoc networks: problems, algorithms, and protocols , 2005, IEEE Communications Magazine.

[13]  J.-C. Renaud,et al.  Multi-Agent Systems on Sensor Networks: A Distributed Reinforcement Learning Approach , 2005, 2005 International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[14]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[15]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[16]  K. J. Ray Liu,et al.  Multinode Cooperative Communications in Wireless Networks , 2007, IEEE Transactions on Signal Processing.

[17]  Aria Nosratinia,et al.  Cooperative communication in wireless networks , 2004, IEEE Communications Magazine.

[18]  Victor C. M. Leung,et al.  Reliable and energy-efficient routing protocol in dense wireless sensor networks , 2008, Int. J. Sens. Networks.

[19]  A. Varga,et al.  THE OMNET++ DISCRETE EVENT SIMULATION SYSTEM , 2003 .

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Manuela M. Veloso,et al.  Multiagent Systems: A Survey from a Machine Learning Perspective , 2000, Auton. Robots.

[22]  Andrew W. Moore,et al.  Distributed Value Functions , 1999, ICML.

[23]  C.-C. Jay Kuo,et al.  Cooperative Communications in Resource-Constrained Wireless Networks , 2007, IEEE Signal Processing Magazine.

[24]  Qian Zhang,et al.  Cross-Layer Design for QoS Support in Multihop Wireless Networks , 2008, Proceedings of the IEEE.

[25]  Pei Liu,et al.  Cooperative wireless communications: a cross-layer approach , 2006, IEEE Wireless Communications.

[26]  Marco Conti,et al.  Cooperation issues in mobile ad hoc networks , 2004, 24th International Conference on Distributed Computing Systems Workshops, 2004. Proceedings..

[27]  K. J. Ray Liu,et al.  Near-optimal reinforcement learning framework for energy-aware sensor communications , 2005, IEEE Journal on Selected Areas in Communications.

[28]  Aria Nosratinia,et al.  Distributed protocols for user cooperation in multi-user wireless networks , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[29]  Victor C. M. Leung,et al.  A New QoS Provisioning Method for Adaptive Multimedia in Wireless Networks , 2008, IEEE Transactions on Vehicular Technology.

[30]  JAMAL N. AL-KARAKI,et al.  Routing techniques in wireless sensor networks: a survey , 2004, IEEE Wireless Communications.

[31]  Bart De Schutter,et al.  Multi-Agent Reinforcement Learning: A Survey , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[32]  Shuguang Cui,et al.  Energy-Efficient Cooperative Communication Based on Power Control and Selective Single-Relay in Wireless Sensor Networks , 2008, IEEE Transactions on Wireless Communications.

[33]  Elza Erkip,et al.  User cooperation diversity. Part I. System description , 2003, IEEE Trans. Commun..

[34]  Xuedong Liang,et al.  Multi-hop mesh cooperative structure based data dissemination for wireless sensor networks , 2009, 2009 11th International Conference on Advanced Communication Technology.