Foresighted digital twin for situational agent selection in production control

Abstract As intelligent Data Acquisition and Analysis in Manufacturing nears its apex, a new era of Digital Twins is dawning. Foresighted Digital Twins enable short- to medium-term system behavior predictions to infer optimal production operation strategies. Creating up-to-the-minute Digital Twins requires both the availability of real-time data and its incorporation and serve as a stepping-stone into developing unprecedented forms of production control. Consequently, we regard a new concept of Digital Twins that includes foresight, thereby enabling situational selection of production control agents. One critical element for adequate system predictions is human behavior as it is neither rule-based nor deterministic, which we therefore model applying Reinforcement Learning. Owing to these ever-changing circumstances, rigid operation strategies crucially restrain reactions, as opposed to circumstantial control strategies that hence can outperform traditional approaches. Building on enhanced foresights we show the superiority of this approach and present strategies for improved situational agent selection.

[1]  Gisela Lanza,et al.  Reinforcement learning for opportunistic maintenance optimization , 2018, Prod. Eng..

[2]  Rainer Stark,et al.  Innovations in digital modelling for next generation manufacturing system design , 2017 .

[3]  Peter Nyhuis,et al.  Changeable Manufacturing - Classification, Design and Operation , 2007 .

[4]  Przemysław Zawadzki,et al.  Smart product design and production control for effective mass customization in the Industry 4.0 concept , 2016 .

[5]  Luca Fumagalli,et al.  Flexible Automation and Intelligent Manufacturing , FAIM 2017 , 27-30 June 2017 , Modena , Italy A review of the roles of Digital Twin in CPS-based production systems , 2017 .

[6]  Simon M. Lucas,et al.  Fast Evolutionary Adaptation for Monte Carlo Tree Search , 2014, EvoApplications.

[7]  Y. Loewenstein,et al.  Reinforcement learning and human behavior , 2014, Current Opinion in Neurobiology.

[8]  Michèle Sebag,et al.  Pilot, Rollout and Monte Carlo Tree Search Methods for Job Shop Scheduling , 2012, LION.

[9]  Felix T.S. Chan,et al.  Defining a Digital Twin-based Cyber-Physical Production System for autonomous manufacturing in smart shop floors , 2019, Int. J. Prod. Res..

[10]  Jianhua Liu,et al.  Digital twin-based smart production management and control framework for the complex product assembly shop-floor , 2018, The International Journal of Advanced Manufacturing Technology.

[11]  G. Fernández,et al.  Reinforcement Learning Signal Predicts Social Conformity , 2009, Neuron.

[12]  Y. Ho,et al.  Simple Explanation of the No-Free-Lunch Theorem and Its Implications , 2002 .

[13]  Rolf Steinhilper,et al.  The Digital Twin: Realizing the Cyber-Physical Production System for Industry 4.0☆ , 2017 .

[14]  Xiaojun Liu,et al.  Dynamic Evaluation Method of Machining Process Planning Based on Digital Twin , 2019, IEEE Access.

[15]  Wilfried Sihn,et al.  Digital Twin in manufacturing: A categorical literature review and classification , 2018 .

[16]  P. Fettke,et al.  Industry 4.0 , 2014, Bus. Inf. Syst. Eng..

[17]  Andreas Kuhnle,et al.  Reinforcement learning for adaptive order dispatching in the semiconductor industry , 2018 .

[18]  Günther Schuh,et al.  Keine Industrie 4.0 ohne den Digitalen Schatten , 2016 .

[19]  Justin L. Gardner,et al.  Learning to Simulate Others' Decisions , 2012, Neuron.

[20]  Alex Pentland,et al.  Modeling and Prediction of Human Behavior , 1999, Neural Computation.

[21]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[22]  Otthein Herzog,et al.  Monte-Carlo Tree Search for Logistics , 2016 .

[23]  Thomas Bauernhansl,et al.  The Digital Shadow of production – A concept for the effective and efficient information supply in dynamic industrial environments , 2018 .

[24]  Jeffrey B. Arthur,et al.  Effects of human resource systems on manufacturing performance and turnover , 1994 .

[25]  Andreas Kuhnle,et al.  Design, Implementation and Evaluation of Reinforcement Learning for an Adaptive Order Dispatching in Job Shop Manufacturing Systems , 2019, Procedia CIRP.

[26]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[27]  William L. Berry,et al.  Approaches to mass customization: configurations and empirical validation , 2000 .