Real-time vision-based multiple object tracking of a production process: Industrial digital twin case study

The adoption of Industry 4.0 technologies within the manufacturing and process industries is widely accepted to have benefits for production cycles, increase system flexibility and give production managers more options on the production line through reconfigurable systems. A key enabler in Industry 4.0 technology is the rise in Cyber-Physical Systems (CPS) and Digital Twins (DTs). Both technologies connect the physical to the cyber world in order to generate smart manufacturing capabilities. State of the art research accurately describes the frameworks, challenges and advantages surrounding these technologies but fails to deliver on testbeds and case studies that can be used for development and validation. This research demonstrates a novel proof of concept Industry 4.0 production system which lays the foundations for future research in DT technologies, process optimisation and manufacturing data analytics. Using a connected system of commercial off-the-shelf cameras to retrofit a standard programmable logic controlled production process, a digital simulation is updated in real time to create the DT. The system can identify and accurately track the product through the production cycle whilst updating the DT in real-time. The implemented system is a lightweight, low cost, customable and scalable design solution which provides a testbed for practical Industry 4.0 research both for academic and industrial research purposes.

[1]  Gang Zhao,et al.  Ontology-based modeling of part digital twin oriented to assembly , 2020, Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture.

[2]  Jinwoo Park,et al.  Developing performance measurement system for Internet of Things and smart factory environment , 2017, Int. J. Prod. Res..

[3]  Hoda A. ElMaraghy,et al.  Learning Integrated Product and Manufacturing Systems , 2015 .

[4]  Jay Lee,et al.  A Cyber-Physical Systems architecture for Industry 4.0-based manufacturing systems , 2015 .

[5]  Marko Mladineo,et al.  Integration and testing of the RFID-enabled Smart Factory concept within the Learning Factory , 2019 .

[6]  Soundar R. T. Kumara,et al.  Cyber-physical systems in manufacturing , 2016 .

[7]  WebbReis Programmable Logic Controllers , 2015 .

[8]  Wilfried Sihn,et al.  Tangible Industry 4.0: A Scenario-Based Approach to Learning for the Future of Production , 2016 .

[9]  Octavian Morariu,et al.  Redundancy and scalability for virtualized MES systems with programmable infrastructure , 2016, Comput. Ind..

[10]  Wolfgang Mahnke,et al.  OPC UA - Service-oriented Architecture for Industrial Applications , 2006, Softwaretechnik-Trends.

[11]  Xun Xu,et al.  A Cyber-Physical Machine Tools Platform using OPC UA and MTConnect , 2019, Journal of Manufacturing Systems.

[12]  Detlef Zuehlke,et al.  SmartFactory – from Vision to Reality in Factory Technologies , 2008 .

[13]  Wang Lu,et al.  Active Shop Scheduling Of Production Process Based On RFID Technology , 2016 .

[14]  Chaoyang Zhang,et al.  A resource-oriented middleware in a prototype cyber-physical manufacturing system , 2018 .

[15]  Vincent Havard,et al.  Data architecture and model design for Industry 4.0 components integration in cyber-physical production systems , 2020 .

[16]  Xiaoqing Frank Liu,et al.  Design and Implementation of Cyber-Physical Manufacturing Cloud Using MTConnect , 2016 .

[17]  P Smitha,et al.  An Efficient Tracking of Multi Object Visual Motion using Hungarian Method , 2015 .

[18]  Lihui Wang,et al.  Current status and advancement of cyber-physical systems in manufacturing , 2015 .

[19]  Eberhard Abele,et al.  Learning factories for future oriented research and education in manufacturing , 2017 .

[20]  Zhengyi Song,et al.  Performance analysis of CyberManufacturing Systems , 2019 .

[21]  Marko Mladineo,et al.  Performance analysis of the RFID system for optimal design of the intelligent assembly line in the learning factory , 2018 .

[22]  Felix T.S. Chan,et al.  Defining a Digital Twin-based Cyber-Physical Production System for autonomous manufacturing in smart shop floors , 2019, Int. J. Prod. Res..

[23]  Mats Björkman,et al.  Transitioning From Standard Automation Solutions to Cyber-Physical Production Systems: An Assessment of Critical Conceptual and Technical Challenges , 2018, IEEE Systems Journal.

[24]  U. Wagner,et al.  The state-of-the-art and prospects of learning factories , 2012 .

[25]  Prashant M. Ambad,et al.  Industry 4.0 – A Glimpse , 2018 .

[26]  Lei Yang,et al.  Fusing RFID and computer vision for fine-grained object tracking , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[27]  Ray Y. Zhong,et al.  Intelligent Manufacturing in the Context of Industry 4.0: A Review , 2017 .

[28]  Widyawardana Adiprawita,et al.  Kalman filter and Iterative-Hungarian Algorithm implementation for low complexity point tracking as part of fast multiple object tracking system , 2016, 2016 6th International Conference on System Engineering and Technology (ICSET).

[29]  Ole Madsen,et al.  The AAU Smart Production Laboratory for Teaching and Research in Emerging Digital Manufacturing Technologies , 2017 .

[30]  Octavian Morariu,et al.  Manufacturing Service Bus Integration Model for Highly Flexible and Scalable Manufacturing Systems , 2013, Service Orientation in Holonic and Multi Agent Manufacturing and Robotics.

[31]  Ray Y. Zhong,et al.  RFID-enabled real-time manufacturing execution system for mass-customization production , 2013 .

[32]  Gábor Erdős,et al.  The MTA SZTAKI Smart Factory: Platform for Research and Project-oriented Skill Development in Higher Education , 2016 .

[33]  Jiahao Wang,et al.  RFID Assisted Object Tracking for Automating Manufacturing Assembly Lines , 2007, IEEE International Conference on e-Business Engineering (ICEBE'07).

[34]  Rolf Steinhilper,et al.  The Digital Twin: Realizing the Cyber-Physical Production System for Industry 4.0☆ , 2017 .

[35]  Siddhartha Kumar Khaitan,et al.  Design Techniques and Applications of Cyberphysical Systems: A Survey , 2015, IEEE Systems Journal.

[36]  Carlos Eduardo Pereira,et al.  Digital Twin Data Modeling with AutomationML and a Communication Methodology for Data Exchange , 2016 .

[37]  Rolf Steinhilper,et al.  The Digital Twin: Demonstrating the Potential of Real Time Data Acquisition in Production Systems ☆ , 2017 .

[38]  Ching-Sheng Wang,et al.  RFID & vision based indoor positioning and identification system , 2011, 2011 IEEE 3rd International Conference on Communication Software and Networks.

[39]  Octavian Morariu,et al.  Manufacturing Service Bus Integration Model for Implementing Highly Flexible and Scalable Manufacturing Systems , 2012 .

[40]  Yu Guo,et al.  Radio frequency identification–enabled monitoring and evaluating in the discrete manufacturing process , 2017 .

[41]  David Romero,et al.  Smart manufacturing: Characteristics, technologies and enabling factors , 2019 .