Adaptive packet scheduling in IoT environment based on Q-learning

Abstract In the Internet of Things (IoT) environment consisting of various devices the arrival rate of data packets dynamically changes. Failure to process them in complying with the QoS requirement can significantly degrade the reliability and quality of the system. Therefore, the gateway collecting the data needs to quickly establish a new scheduling policy according to the changing traffic condition. The existing packet scheduling schemes are not effective for IoT since the data transmission pattern is not defined in advance. Q-learning is a type of reinforcement learning that can establish a dynamic scheduling policy according to the state of each queue without any prior knowledge on the network status. In this paper a novel Q-learning scheme is proposed which updates the Q-table and reward table based on the condition of the queues in the gateway and adjusts the reward value according to the time slot. Computer simulation reveals that the proposed scheme significantly reduces the scheduling time while allowing high accuracy compared to the existing Q-learning scheme based on Stochastic Learning Automaton (SLA).

[1]  Roberto Rojas-Cessa,et al.  Greedy scheduling of tasks with time constraints for energy-efficient cloud-computing data centers , 2015, Journal of Cloud Computing.

[2]  Jamal Shahrabi,et al.  A reinforcement learning approach to parameter estimation in dynamic job shop scheduling , 2017, Comput. Ind. Eng..

[3]  Lyes Khoukhi,et al.  Industrial IoT Data Scheduling Based on Hierarchical Fog Computing: A Key for Enabling Smart Factory , 2018, IEEE Transactions on Industrial Informatics.

[4]  Tarik Taleb,et al.  Dynamic Multilevel Priority Packet Scheduling Scheme for Wireless Sensor Network , 2013, IEEE Transactions on Wireless Communications.

[5]  Xiaomin Zhu,et al.  Towards energy-efficient scheduling for real-time tasks under uncertain cloud computing environment , 2015, J. Syst. Softw..

[6]  Hwee Pink Tan,et al.  Markov Decision Processes With Applications in Wireless Sensor Networks: A Survey , 2015, IEEE Communications Surveys & Tutorials.

[7]  Ling Li,et al.  QoS-Aware Scheduling of Services-Oriented Internet of Things , 2014, IEEE Transactions on Industrial Informatics.

[8]  Navin Kumar,et al.  Packet Scheduling Scheme to Guarantee QoS in Internet of Things , 2018, Wirel. Pers. Commun..

[9]  Kok-Lim Alvin Yau,et al.  Reinforcement learning for context awareness and intelligence in wireless networks: Review, new features and open issues , 2012, J. Netw. Comput. Appl..

[10]  Mohammed Atiquzzaman,et al.  Scheduling internet of things applications in cloud computing , 2016, Annals of Telecommunications.

[11]  Florin Pop,et al.  New scheduling approach using reinforcement learning for heterogeneous distributed systems , 2017, J. Parallel Distributed Comput..

[12]  Gang Feng,et al.  Resource Allocation in Software Defined Wireless Networks , 2017, IEEE Network.

[13]  Weiwei Lin,et al.  Random task scheduling scheme based on reinforcement learning in cloud computing , 2015, Cluster Computing.

[14]  Marco Wiering,et al.  Comparing exploration strategies for Q-learning in random stochastic mazes , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).