Deep reinforcement learning-based dynamic scheduling in smart manufacturing

Abstract Scheduling problems are a classic type of optimization problems in the manufacturing domain, such as job shop scheduling, flexible job shop scheduling, and distributed job shop scheduling problems. Especially, the dynamic task scheduling problem is closer to the requirements of real manufacturing systems than the static scheduling problem. In recent years, with the deeper application of the Internet of Things, big data, and cloud platform technologies, manufacturing systems have evolved from job shops to networked, collaborative and intelligent manufacturing systems. Smart manufacturing scheduling has some characteristics compared with job shop scheduling not only because of the larger number of tasks and services, but also because of the dynamic state of services and uncertainties. In this paper, we analyze the smart manufacturing service scheduling problem and give its mathematical description. Then a deep reinforcement learning-based method is proposed to minimize the maximum completion time of all tasks. In the system framework of the proposed method, the agent, environment, as well as the interaction between them, are designed. The queue times of all candidate services are considered as the system state, and the maximum queue time at the current moment is considered as the target value. Besides, we use two networks the prediction network and the target network to learn the prediction value and the target value, respectively. Two case studies are used to show the efficiency of the considered problem and the proposed method.