Use of Q-learning approaches for practical medium access control in wireless sensor networks

This paper studies the potential of a novel approach to ensure more efficient and intelligent assignment of capacity through medium access control (MAC) in practical wireless sensor networks. Q-Learning is employed as an intelligent transmission strategy. We review the existing MAC protocols in the context of Q-learning. A recently-proposed, ALOHA and Q-Learning based MAC scheme, ALOHA-Q, is considered which improves the channel performance significantly with a key benefit of simplicity. Practical implementation issues of ALOHA-Q are studied. We demonstrate the performance of the ALOHA-Q through extensive simulations and evaluations in various testbeds. A new exploration/exploitation method is proposed to strengthen the merits of the ALOHA-Q against dynamic the channel and environment conditions.

[1]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[2]  Yan Yan,et al.  Adaptation of the ALOHA-Q protocol to Multi-Hop Wireless Sensor Networks , 2014 .

[3]  David Grace,et al.  ALOHA and Q-Learning based medium access control for Wireless Sensor Networks , 2012, 2012 International Symposium on Wireless Communication Systems (ISWCS).

[4]  Giancarlo Fortino,et al.  QL-MAC: A Q-Learning Based MAC for Wireless Sensor Networks , 2013, ICA3PP.

[5]  Ian F. Akyildiz,et al.  Sensor Networks , 2002, Encyclopedia of GIS.

[6]  Deborah Estrin,et al.  An energy-efficient MAC protocol for wireless sensor networks , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[7]  Ann Nowé,et al.  Decentralised reinforcement learning for energy-efficient scheduling in wireless sensor networks , 2012, Int. J. Commun. Networks Distributed Syst..

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  Özlem Durmaz Incel,et al.  QoS-aware MAC protocols for wireless sensor networks: A survey , 2011, Comput. Networks.

[10]  Zhidong Deng,et al.  Distributed self-learning scheduling approach for wireless sensor network , 2013, Ad Hoc Networks.

[11]  Injong Rhee,et al.  Z-MAC: a hybrid MAC for wireless sensor networks , 2005, SenSys '05.

[12]  Praveen Budhwar,et al.  TinyOS: An Operating System for Wireless Sensor Networks , 2015 .

[13]  Zhenzhen Liu,et al.  RL-MAC: A QoS-Aware Reinforcement Learning based MAC Protocol for Wireless Sensor Networks , 2006, 2006 IEEE International Conference on Networking, Sensing and Control.

[14]  Tim Clarke,et al.  Practical Implementation Issues of Reinforcement Learning Based ALOHA for Wireless Sensor Networks , 2013, ISWCS.

[15]  Tim Clarke,et al.  Distributed Frame Size Selection for a Q learning based Slotted ALOHA Protocol , 2013, International Symposium on Wireless Communication Systems.

[16]  Cem Ersoy,et al.  MAC protocols for wireless sensor networks: a survey , 2006, IEEE Communications Magazine.

[17]  Peng Ning,et al.  Secure Code Dissemination in Wireless Sensor Networks , 2011, Encyclopedia of Cryptography and Security.

[18]  Zhenzhen Liu,et al.  RL-MAC: a reinforcement learning based MAC protocol for wireless sensor networks , 2006, Int. J. Sens. Networks.