A Real-Time Distance Measurement System for a Digital Twin Using Mixed Reality Goggles

This paper presents a new system architecture for controlling industrial devices using Mixed Reality (MR) applications and a new method based upon them for measuring the distance between real and virtual points. The research has been carried out using a physical robot and its Digital Twin (DT). The possibility of controlling them using gestures recognized by Mixed Reality goggles has been presented. The extension of the robot’s environment with a 3D model capable of following its movements and positions was also analyzed. The system was supervised by an industrial Programmable Logic Controller (PLC) serving as an end point for the data sent by the goggles and controlling the movements of the real robot by activating the corresponding control. The results of the preliminary measurements presented here concerned the responsiveness of the system and showing the influence of system parameters in the accuracy of distance estimation between measured points.

[1]  I. Dominik,et al.  Application of self-oscillating system for stress measurement in metal , 2012 .

[2]  Kerim Kürşat ÇEVİK,et al.  Computer Vision Based Distance Measurement System using Stereo Camera View , 2019, 2019 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT).

[3]  Kourosh Khoshelham,et al.  Indoor Mapping Eyewear: Geometric Evaluation of Spatial Mapping Capability of Hololens , 2019 .

[4]  Martin Gaedke,et al.  NAND-measure: An Android App for Marker-based Spatial Measurement , 2021, VISIGRAPP.

[5]  Jesús Morales,et al.  Analysis of 3D Scan Measurement Distribution with Application to a Multi-Beam Lidar on a Rotating Platform , 2018, Sensors.

[6]  Peter Dieckmann,et al.  Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review , 2020, International journal of medical education.

[7]  Jun-Ho Huh PLC-Integrated Sensing Technology in Mountain Regions for Drone Landing Sites: Focusing on Software Technology , 2018, Sensors.

[8]  Carlos Eduardo Pereira,et al.  Visualising the digital twin using web services and augmented reality , 2016, 2016 IEEE 14th International Conference on Industrial Informatics (INDIN).

[9]  Abdulmotaleb El-Saddik,et al.  Technical Evaluation of HoloLens for Multimedia: A First Look , 2018, IEEE MultiMedia.

[11]  Ireneusz Dominik,et al.  Energy Harvesting System Based on Ionic Polymer-Metal Composites - Identification of Electrical Parameters , 2014 .

[12]  Monica Carfagni,et al.  On the Performance of the Intel SR30 Depth Camera: Metrological and Critical Characterization , 2017, IEEE Sensors Journal.

[13]  Hiroyuki Tomiyama,et al.  A Neck-Floor Distance Analysis-Based Fall Detection System Using Deep Camera , 2021 .

[14]  Saurabh Sinha,et al.  A Review of Industry 4.0 Manufacturing Process Security Risks , 2019, Applied Sciences.

[15]  Vehbi C. Gungor,et al.  Smart grid communication and information technologies in the perspective of Industry 4.0: Opportunities and challenges , 2018, Comput. Sci. Rev..

[16]  Kunsoo Huh,et al.  A stereo vision-based obstacle detection system in vehicles , 2008 .

[17]  I. Dominik,et al.  Ionic polymer-metal composite displacement sensors , 2016 .

[18]  Jae Yeol Lee,et al.  An integrated mixed reality system for safety-aware human-robot collaboration using deep learning and digital twin generation , 2022, Robotics Comput. Integr. Manuf..

[19]  Marco Conti,et al.  Emerging Trends in Hybrid Wireless Communication and Data Management for the Industry 4.0 , 2018 .

[20]  Louise Wright,et al.  How to tell the difference between a model and a digital twin , 2020, Advanced Modeling and Simulation in Engineering Sciences.

[21]  Enzo Morosini Frazzon,et al.  Solving the Job-Shop Scheduling Problem in the Industry 4.0 Era , 2018, Technologies.

[22]  Tiago M. Fernández-Caramés,et al.  A Review on Industrial Augmented Reality Systems for the Industry 4.0 Shipyard , 2018, IEEE Access.

[23]  Bruno Simões,et al.  X-Reality System Architecture for Industry 4.0 Processes , 2018, Multimodal Technol. Interact..

[24]  K. Weimer,et al.  Mixed Reality with HoloLens: Where Virtual Reality Meets Augmented Reality in the Operating Room , 2017, Plastic and reconstructive surgery.

[25]  T. Tonnesen,et al.  Self-Excited Acoustical System Frequency Monitoring for Refractory Concrete under Uniaxial Compression , 2021, Energies.

[26]  Xun Xu,et al.  Visualisation of the Digital Twin data in manufacturing by using Augmented Reality , 2019, Procedia CIRP.

[27]  Jerzy Stojek,et al.  Use of the K-Nearest Neighbour Classifier in Wear Condition Classification of a Positive Displacement Pump , 2021, Sensors.

[28]  Borko Furht,et al.  Handbook of Augmented Reality , 2011 .

[29]  Luigi Pomante,et al.  SPOF—Slave Powerlink on FPGA for Smart Sensors and Actuators Interfacing for Industry 4.0 Applications , 2019 .

[30]  Christian Ramsauer,et al.  An Industry 4.0 Production Workplace Enhanced by Using Mixed Reality Assembly Instructions with Microsoft HoloLens , 2020, MuC Workshopband.

[31]  Samad M. E. Sepasgozar,et al.  Digital Twin and Web-Based Virtual Gaming Technologies for Online Education: A Case of Construction Management and Engineering , 2020, Applied Sciences.

[32]  Ireneusz Dominik,et al.  Integrated stress measurement system in tower crane mast , 2017 .

[33]  Dominik Ireneusz,et al.  Preliminary signal filtering in Self-Excited Acoustical System for stress change measurement , 2013, Proceedings of the 32nd Chinese Control Conference.