Reinforcement Learning for Multiple Access Control in Wireless Sensor Networks: Review, Model, and Open Issues

Wireless sensor networking is a viable communication technology among low-cost and energy-limited sensor nodes deployed in an environment. Due to high operational features, the application area of this technology is extended significantly but with some energy related challenges. One main cause of the nodes energy wasting in these networks is idle listening characterized with no communication activity. This drawback can be mitigated by the means of energy-efficient multiple access control schemes so as to minimize idle listening. In this paper, we discuss the applicability of distributed learning algorithms namely reinforcement learning towards multiple access control (MAC) in wireless sensor networks. We perform a comparative review of relevant work in the literature and then present a cooperative multi agent reinforcement learning framework for MAC design in wireless sensor networks. Accordingly, the paper concludes with some major challenges and open issues of distributed MAC design using reinforcement learning.

[1]  Kok-Lim Alvin Yau,et al.  Reinforcement learning for context awareness and intelligence in wireless networks: Review, new features and open issues , 2012, J. Netw. Comput. Appl..

[2]  Dong In Kim,et al.  Game Theoretic Approaches for Multiple Access in Wireless Networks: A Survey , 2011, IEEE Communications Surveys & Tutorials.

[3]  Deborah Estrin,et al.  Medium access control with coordinated adaptive sleeping for wireless sensor networks , 2004, IEEE/ACM Transactions on Networking.

[4]  Ganesh K. Venayagamoorthy,et al.  Computational Intelligence in Wireless Sensor Networks: A Survey , 2011, IEEE Communications Surveys & Tutorials.

[5]  H. Vincent Poor,et al.  Distributed learning in wireless sensor networks , 2005, IEEE Signal Processing Magazine.

[6]  Krishna M. Sivalingam,et al.  Reinforcement Learning Based Geographic Routing Protocol for UWB Wireless Sensor Network , 2007, IEEE GLOBECOM 2007 - IEEE Global Telecommunications Conference.

[7]  Zhidong Deng,et al.  Distributed self-learning scheduling approach for wireless sensor network , 2013, Ad Hoc Networks.

[8]  A. Forstert,et al.  FROMS: Feedback Routing for Optimizing Multiple Sinks in WSN with Reinforcement Learning , 2007, 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information.

[9]  K. J. Ray Liu,et al.  Near-optimal reinforcement learning framework for energy-aware sensor communications , 2005, IEEE Journal on Selected Areas in Communications.

[10]  Djamal Zeghlache,et al.  A distributed reinforcement learning approach to maximize resource utilization and control handover dropping in multimedia wireless networks , 2002, The 13th IEEE International Symposium on Personal, Indoor and Mobile Radio Communications.

[11]  Leslie Pack Kaelbling,et al.  Mobilized ad-hoc networks: a reinforcement learning approach , 2004 .

[12]  Prashant J. Shenoy,et al.  An adaptive link layer for heterogeneous multi-radio mobile sensor networks , 2010, IEEE Journal on Selected Areas in Communications.

[13]  Zhenzhen Liu,et al.  RL-MAC: a reinforcement learning based MAC protocol for wireless sensor networks , 2006, Int. J. Sens. Networks.

[14]  Ian F. Akyildiz,et al.  Wireless sensor networks: a survey , 2002, Comput. Networks.

[15]  Ian F. Akyildiz,et al.  Wireless sensor networks , 2007 .

[16]  Koen Langendoen,et al.  An adaptive energy-efficient MAC protocol for wireless sensor networks , 2003, SenSys '03.

[17]  Li Xiao,et al.  The Evolution of MAC Protocols in Wireless Sensor Networks: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[18]  Ian F. Akyildiz,et al.  Wireless Sensor Networks: Akyildiz/Wireless Sensor Networks , 2010 .

[19]  David Tse,et al.  Mobility increases the capacity of ad hoc wireless networks , 2002, TNET.

[20]  Cem Ersoy,et al.  MAC protocols for wireless sensor networks: a survey , 2006, IEEE Communications Magazine.

[21]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[22]  Michael I. Jordan,et al.  Nonparametric decentralized detection using kernel methods , 2005, IEEE Transactions on Signal Processing.

[23]  H. Vincent Poor,et al.  Consistency in models for distributed learning under communication constraints , 2005, IEEE Transactions on Information Theory.

[24]  Ann Nowé,et al.  Decentralized Learning in Wireless Sensor Networks , 2009, ALA.

[25]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[26]  Kok-Lim Alvin Yau,et al.  Enhancing network performance in Distributed Cognitive Radio Networks using single-agent and multi-agent Reinforcement Learning , 2010, IEEE Local Computer Network Conference.

[27]  Abdelhakim Hafid,et al.  High accuracy localization method using AoA in sensor networks , 2009, Comput. Networks.

[28]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[29]  Victor C. M. Leung,et al.  A New QoS Provisioning Method for Adaptive Multimedia in Wireless Networks , 2008, IEEE Transactions on Vehicular Technology.