Optimization of global production scheduling with deep reinforcement learning

Abstract Industrie 4.0 introduces decentralized, self-organizing and self-learning systems for production control. At the same time, new machine learning algorithms are getting increasingly powerful and solve real world problems. We apply Google DeepMind’s Deep Q Network (DQN) agent algorithm for Reinforcement Learning (RL) to production scheduling to achieve the Industrie 4.0 vision for production control. In an RL environment cooperative DQN agents, which utilize deep neural networks, are trained with user-defined objectives to optimize scheduling. We validate our system with a small factory simulation, which is modeling an abstracted frontend-of-line semiconductor production facility.

[1]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[2]  Wei Zhang,et al.  A Reinforcement Learning Approach to job-shop Scheduling , 1995, IJCAI.

[3]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[4]  Wilfried Brauer,et al.  Multi-machine scheduling-a multi-agent learning approach , 1998, Proceedings International Conference on Multi Agent Systems (Cat. No.98EX160).

[5]  Sanja Petrovic,et al.  SURVEY OF DYNAMIC SCHEDULING IN MANUFACTURING SYSTEMS , 2006 .

[6]  Sean Luke,et al.  Cooperative Multi-Agent Learning: The State of the Art , 2005, Autonomous Agents and Multi-Agent Systems.

[7]  Tapas K. Das,et al.  A multi-agent reinforcement learning approach to obtaining dynamic control policies for stochastic lot scheduling problem , 2005, Simul. Model. Pract. Theory.

[8]  Martin A. Riedmiller,et al.  A Neural Reinforcement Learning Approach to Learn Local Dispatching Policies in Production Scheduling , 1999, IJCAI.

[9]  Jie Wang,et al.  Optimized Adaptive Scheduling of a Manufacturing Process System with Multi-skill Workforce and Multiple Machine Types: An Ontology-based, Multi-agent Reinforcement Learning Approach , 2016 .

[10]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[11]  Rolf Steinhilper,et al.  The Digital Twin: Realizing the Cyber-Physical Production System for Industry 4.0☆ , 2017 .

[12]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[13]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Thomas G. Dietterich,et al.  High-Performance Job-Shop Scheduling With A Time-Delay TD(λ) Network , 1995, NIPS 1995.

[15]  Alexander Mordvintsev,et al.  Inceptionism: Going Deeper into Neural Networks , 2015 .