Smart Manufacturing Scheduling With Edge Computing Using Multiclass Deep Q Network

Manufacturing is involved with complex job shop scheduling problems (JSP). In smart factories, edge computing supports computing resources at the edge of production in a distributed way to reduce response time of making production decisions. However, most works on JSP did not consider edge computing. Therefore, this paper proposes a smart manufacturing factory framework based on edge computing, and further investigates the JSP under such a framework. With recent success of some AI applications, the deep Q network (DQN), which combines deep learning and reinforcement learning, has showed its great computing power to solve complex problems. Therefore, we adjust the DQN with an edge computing framework to solve the JSP. Different from the classical DQN with only one decision, this paper extends the DQN to address the decisions of multiple edge devices. Simulation results show that the proposed method performs better than the other methods using only one dispatching rule.

[1]  Mehmet Emin Aydin,et al.  Dynamic job-shop scheduling using reinforcement learning agents , 2000, Robotics Auton. Syst..

[2]  S. S. Panwalkar,et al.  A Survey of Scheduling Rules , 1977, Oper. Res..

[3]  Mohsen Guizani,et al.  Performance Modeling of Representative Load Sharing Schemes for Clustered Servers in Multiaccess Edge Computing , 2019, IEEE Internet of Things Journal.

[4]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[5]  Mengjie Zhang,et al.  Surrogate-Assisted Genetic Programming With Simplified Models for Automated Design of Dispatching Rules , 2017, IEEE Transactions on Cybernetics.

[6]  Reha Uzsoy,et al.  Benchmarks for shop scheduling problems , 1998, Eur. J. Oper. Res..

[7]  Guangjie Han,et al.  Edge Computing-Based Intelligent Manhole Cover Management System for Smart Cities , 2018, IEEE Internet of Things Journal.

[8]  Wenxin Liu,et al.  A neural network model and algorithm for the hybrid flow shop scheduling problem in a dynamic environment , 2005, J. Intell. Manuf..

[9]  Guangjie Han,et al.  Socialized healthcare service recommendation using deep learning , 2018, Neural Computing and Applications.

[10]  Hui Tian,et al.  Selective Offloading in Mobile Edge Computing for the Green Internet of Things , 2018, IEEE Network.

[11]  Ke Zhang,et al.  Delay constrained offloading for Mobile Edge Computing in cloud-enabled vehicular networks , 2016, 2016 8th International Workshop on Resilient Networks Design and Modeling (RNDM).

[12]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[13]  Sergey Levine,et al.  Continuous Deep Q-Learning with Model-based Acceleration , 2016, ICML.

[14]  Shijun Liu,et al.  DRL-Scheduling: An Intelligent QoS-Aware Job Scheduling Framework for Applications in Clouds , 2018, IEEE Access.

[15]  Chu-Sing Yang,et al.  Job shop scheduling based on ACO with a hybrid solution construction strategy , 2011, 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011).

[16]  Chao-Ton Su,et al.  Real-time scheduling for a smart factory using a reinforcement learning approach , 2018, Comput. Ind. Eng..

[17]  Guangjie Han,et al.  A Maximum Cache Value Policy in Hybrid Memory-Based Edge Computing for Mobile Devices , 2019, IEEE Internet of Things Journal.

[18]  Jan Karel Lenstra,et al.  Job Shop Scheduling by Simulated Annealing , 1992, Oper. Res..

[19]  Yuping Wang,et al.  A new hybrid genetic algorithm for job shop scheduling problem , 2012, Comput. Oper. Res..

[20]  Yoke San Wong,et al.  Dynamic selection of dispatching rules for job shop scheduling , 2000 .

[21]  Wei Zhang,et al.  A Reinforcement Learning Approach to job-shop Scheduling , 1995, IJCAI.

[22]  梁炜,et al.  Neural network and genetic algorithm-based hybrid approach to expanded job-shop scheduling , 2001 .

[23]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[24]  Yanchun Liang,et al.  An Effective PSO and AIS-Based Hybrid Intelligent Algorithm for Job-Shop Scheduling , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[25]  William J. Cook,et al.  A Computational Study of the Job-Shop Scheduling Problem , 1991, INFORMS Journal on Computing.

[26]  Mohammed Atiquzzaman,et al.  Energy efficient device discovery for reliable communication in 5G-based IoT and BSNs using unmanned aerial vehicles , 2017, J. Netw. Comput. Appl..

[27]  Éric D. Taillard,et al.  Benchmarks for basic scheduling problems , 1993 .

[28]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.