Determining node duty cycle using Q-learning and linear regression for WSN

Wireless sensor network (WSN) is effective for monitoring the target environment, which consists of a large number of sensor nodes of limited energy. An efficient medium access control (MAC) protocol is thus imperative to maximize the energy efficiency and performance of WSN. The most existing MAC protocols are based on the scheduling of sleep and active period of the nodes, and do not consider the relationship between the load condition and performance. In this paper a novel scheme is proposed to properly determine the duty cycle of the WSN nodes according to the load, which employs the Q-learning technique and function approximation with linear regression. This allows low-latency energy-efficient scheduling for a wide range of traffic conditions, and effectively overcomes the limitation of Q-learning with the problem of continuous state-action space. NS3 simulation reveals that the proposed scheme significantly improves the throughput, latency, and energy efficiency compared to the existing fully active scheme and S-MAC.

[1]  Shu Du,et al.  RMAC: A Routing-Enhanced Duty-Cycle MAC Protocol for Wireless Sensor Networks , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[2]  Deborah Estrin,et al.  Medium access control with coordinated adaptive sleeping for wireless sensor networks , 2004, IEEE/ACM Transactions on Networking.

[3]  Frank L. Lewis,et al.  Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  Rong Zheng,et al.  On-demand power management for ad hoc networks , 2005, Ad Hoc Networks.

[5]  Koen Langendoen,et al.  An adaptive energy-efficient MAC protocol for wireless sensor networks , 2003, SenSys '03.

[6]  David E. Culler,et al.  Versatile low power media access for wireless sensor networks , 2004, SenSys '04.

[7]  Chunming Qiao,et al.  Medium access control with a dynamic duty cycle for sensor networks , 2004, 2004 IEEE Wireless Communications and Networking Conference (IEEE Cat. No.04TH8733).

[8]  Prasun Sinha,et al.  CMAC: An Energy Efficient MAC Layer Protocol Using Convergent Packet Forwarding for Wireless Sensor Networks , 2007, 2007 4th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks.

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Eric Anderson,et al.  X-MAC: a short preamble MAC protocol for duty-cycled wireless sensor networks , 2006, SenSys '06.

[11]  Rong Zheng,et al.  Asynchronous wakeup for ad hoc networks , 2003, MobiHoc '03.

[12]  Young-Chon Kim,et al.  P-MAC: A Cross-Layer Duty Cycle MAC Protocol Towards Pipelining for Wireless Sensor Networks , 2011, 2011 IEEE International Conference on Communications (ICC).

[13]  Tim Clarke,et al.  Use of Q-learning approaches for practical medium access control in wireless sensor networks , 2016, Eng. Appl. Artif. Intell..

[14]  Omer Gurewitz,et al.  RI-MAC: a receiver-initiated asynchronous duty cycle MAC protocol for dynamic traffic loads in wireless sensor networks , 2008, SenSys '08.

[15]  Mohamed Ibnkahla,et al.  Wireless sensor networks: Applications and challenges , 2007, 2007 9th International Symposium on Signal Processing and Its Applications.

[16]  Nitin H. Vaidya,et al.  An energy efficient MAC protocol for wireless LANs , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[17]  Zhidong Deng,et al.  Distributed self-learning scheduling approach for wireless sensor network , 2013, Ad Hoc Networks.

[18]  Boris Bellalta,et al.  Energy efficiency of MAC protocols in low data rate wireless multimedia sensor networks: A comparative study , 2017, Ad Hoc Networks.

[19]  Deborah Estrin,et al.  An energy-efficient MAC protocol for wireless sensor networks , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.