Review of Digital Twin-based Interaction in Smart Manufacturing: Enabling Cyber-Physical Systems for Human-Machine Interaction

ABSTRACT After its introduction around 20 years ago, the Digital Twin (DT) approach has recently attracted much interest in shaping the next generation of manufacturing. In the last years, many definitions and descriptions of the DT have been published, examining different aspects of its implementation. This paper is the first to present an analysis on the integration and interaction of human and DT in smart manufacturing systems in form of a scoping review following the PRISMA-ScR methodology. It presents the current state of the art of DT-based human-machine interaction (HMI), its implications, and future research directions. Filtering from 278 publications over the last decade, the analysis includes 23 publications, all published from 2016 to 2020. The results show the predominant scenarios and applications of DT-based HMI and identify the current division of labor between human and DT. The paper concludes with an integration of these findings into a human-centered classification of DTs as well as future research directions.

[1]  Xun Xu,et al.  Visualisation of the Digital Twin data in manufacturing by using Augmented Reality , 2019, Procedia CIRP.

[2]  Nadja Hoßbach,et al.  Dimensions of Digital Twin Applications - A Literature Review , 2019, AMCIS.

[3]  Sang Do Noh,et al.  Service-oriented platform for smart operation of dyeing and finishing industry , 2019, Int. J. Comput. Integr. Manuf..

[4]  Xun Xu,et al.  A Cyber-Physical Machine Tools Platform using OPC UA and MTConnect , 2019, Journal of Manufacturing Systems.

[5]  Luca Fumagalli,et al.  Flexible Automation and Intelligent Manufacturing , FAIM 2017 , 27-30 June 2017 , Modena , Italy A review of the roles of Digital Twin in CPS-based production systems , 2017 .

[6]  Tariq Masood,et al.  Virtual reality in manufacturing: immersive and collaborative artificial-reality in design of human-robot workspace , 2019, Int. J. Comput. Integr. Manuf..

[7]  Thies Beinke,et al.  Improving Human-Machine Interaction with a Digital Twin , 2020 .

[8]  I. Graessler,et al.  Integration of a digital twin as human representation in a scheduling procedure of a cyber-physical production system , 2017, 2017 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM).

[9]  Tiziana Catarci,et al.  A Conceptual Architecture and Model for Smart Manufacturing Relying on Service-Based Digital Twins , 2019, 2019 IEEE International Conference on Web Services (ICWS).

[10]  Zhong Fan,et al.  Digital Twin: Enabling Technologies, Challenges and Open Research , 2020, IEEE Access.

[11]  J. McGowan,et al.  PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation , 2018, Annals of Internal Medicine.

[12]  Ashutosh Tiwari,et al.  The effectiveness of virtual environments in developing collaborative strategies between industrial robots and humans , 2019, Robotics and Computer-Integrated Manufacturing.

[13]  Michael W. Grieves,et al.  Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems , 2017 .

[14]  Marco Sacco,et al.  A Telemetry-driven Approach to Simulate Data-intensive Manufacturing Processes , 2016 .

[15]  Olivier Cardin,et al.  Classification of cyber-physical production systems applications: Proposition of an analysis framework , 2018, Comput. Ind..

[16]  Toivo Tähemaa,et al.  DIGITAL TWIN BASED SYNCHRONISED CONTROL AND SIMULATION OF THE INDUSTRIAL ROBOTIC CELL USING VIRTUAL REALITY , 2019, Journal of Machine Engineering.

[17]  Control architecture for digital twin-based human-machine interaction in a novel container unloading system , 2020 .

[18]  Ting Qu,et al.  User acceptance of virtual reality technology for practicing digital twin-based crisis management , 2020, Int. J. Comput. Integr. Manuf..

[19]  Rainer Stark,et al.  Development and operation of Digital Twins for technical systems and services , 2019, CIRP Annals.

[20]  Álvaro Segura,et al.  Sustainable and flexible industrial human machine interfaces to support adaptable applications in the Industry 4.0 paradigm , 2019, Int. J. Prod. Res..

[21]  Xun Xu,et al.  Cloud-based manufacturing equipment and big data analytics to enable on-demand manufacturing services , 2019, Robotics and Computer-Integrated Manufacturing.

[22]  Boris Otto,et al.  A Taxonomy of Digital Twins , 2020, AMCIS.

[23]  Álvaro Segura,et al.  Visual computing technologies to support the Operator 4.0 , 2020, Comput. Ind. Eng..

[24]  Antonio Padovano,et al.  A Digital Twin based Service Oriented Application for a 4.0 Knowledge Navigation in the Smart Factory , 2018 .

[25]  Antonio Padovano,et al.  Ubiquitous knowledge empowers the Smart Factory: The impacts of a Service-oriented Digital Twin on enterprises' performance , 2022, Annu. Rev. Control..

[26]  Raja Parasuraman,et al.  Human-Automation Interaction , 2005 .

[27]  Daniela Fogli,et al.  A Survey on Digital Twin: Definitions, Characteristics, Applications, and Design Implications , 2019, IEEE Access.

[28]  Alois Knoll,et al.  Modular Fault Ascription and Corrective Maintenance Using a Digital Twin , 2018 .

[29]  Gergely Horváth,et al.  Gesture Control of Cyber Physical Systems , 2017 .

[30]  A. Tammaro,et al.  Extending Industrial Digital Twins with Optical Object Tracking , 2017, CEIG.

[31]  He Zhang,et al.  Digital Twin in Industry: State-of-the-Art , 2019, IEEE Transactions on Industrial Informatics.

[32]  James Moyne,et al.  Dynamic Rerouting of Cyber-Physical Production Systems in Response to Disruptions Based on SDC Framework , 2019, 2019 American Control Conference (ACC).

[33]  Elisa Negri,et al.  Review of digital twin applications in manufacturing , 2019, Comput. Ind..

[34]  Sungjoo Kang,et al.  Design and Implementation of Runtime Verification Framework for Cyber-Physical Production Systems , 2019, Journal of Engineering.

[35]  Rachel L. Shaw,et al.  Conducting literature reviews , 2010 .

[36]  Jason Yon,et al.  Characterising the Digital Twin: A systematic literature review , 2020, CIRP Journal of Manufacturing Science and Technology.

[37]  Xiang Wan,et al.  Research on modelling and optimization of hot rolling scheduling , 2018, J. Ambient Intell. Humaniz. Comput..

[38]  M. Frosolini,et al.  Using RFID technology and Discrete-Events, Agent-Based simulation tools to build Digital-Twins of large warehouses , 2019, 2019 IEEE International Conference on RFID Technology and Applications (RFID-TA).

[39]  Sang Do Noh,et al.  Design and implementation of a digital twin application for a connected micro smart factory , 2019, Int. J. Comput. Integr. Manuf..

[40]  Huiyue Dong,et al.  Review of digital twin about concepts, technologies, and industrial applications , 2020 .

[41]  Wilfried Sihn,et al.  Digital Twin in manufacturing: A categorical literature review and classification , 2018 .

[42]  Andrew Y. C. Nee,et al.  Digital Twins and Cyber–Physical Systems toward Smart Manufacturing and Industry 4.0: Correlation and Comparison , 2019, Engineering.

[43]  Arne Bilberg,et al.  Digital twin driven human–robot collaborative assembly , 2019, CIRP Annals.

[44]  M R Endsley,et al.  Level of automation effects on performance, situation awareness and workload in a dynamic control task. , 1999, Ergonomics.

[45]  Yelin Fu,et al.  Digital twins in human understanding: a deep learning-based method to recognize personality traits , 2020, Int. J. Comput. Integr. Manuf..

[46]  George Chryssolouris,et al.  The digital twin implementation for linking the virtual representation of human-based production tasks to their physical counterpart in the factory-floor , 2018, Int. J. Comput. Integr. Manuf..

[47]  Edward H. Glaessgen,et al.  The Digital Twin Paradigm for Future NASA and U.S. Air Force Vehicles , 2012 .

[48]  Gregor Engels,et al.  A Digital Twin-Based Multi-modal UI Adaptation Framework for Assistance Systems in Industry 4.0 , 2019, HCI.

[49]  Pai Zheng,et al.  A systematic design approach for service innovation of smart product-service systems , 2018, Journal of Cleaner Production.

[50]  Detlef Zühlke,et al.  Future Modeling and Simulation of CPS-based Factories: an Example from the Automotive Industry , 2016 .

[51]  Kevin I-Kai Wang,et al.  Digital Twin-driven smart manufacturing: Connotation, reference model, applications and research issues , 2020, Robotics Comput. Integr. Manuf..