Mixed reality-based user interface for quality control inspection of car body surfaces

Abstract In recent years, the quality control of car body surfaces production lines have been put in the context of Industry 4.0. The emergence of automatic defect detection systems have helped to standardize the brand quality and gather information about all quality control tasks performed by workers. However, current worker interfaces used to indicate the location and other characteristics of the defects found by these systems have overcome the ergonomics of workers and increased their stress at work. This paper presents a novel mixed reality-based user interface for quality control inspection which is more intuitive, in order to improve the ergonomics of workers, reduce their stress at work and improve the productivity of current quality control production lines. An experimental prototype is shown in the paper in order to demonstrate the benefits of the proposed interface. In addition, the paper shows the results of several usability tests that compare the proposed mixed reality-based user interface with current interfaces used in important factories such as Mercedes-Benz, analyzing the benefits and drawbacks of each interface.

[1]  Josep Tornero,et al.  Detección de Defectos en Carrocerías de Vehículos Basado en Visión Artificial: Diseño e Implantación , 2012 .

[2]  Thies Pfeiffer,et al.  Comparing Conventional and Augmented Reality Instructions for Manual Assembly Tasks , 2017, PETRA.

[3]  Eleonora Bottani,et al.  Design and Testing of an Augmented Reality Solution to Enhance Operator Safety in the Food Industry , 2018 .

[4]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[5]  Mao-Jiun J. Wang,et al.  Usability evaluation of an instructional application based on Google Glass for mobile phone disassembly tasks. , 2019, Applied ergonomics.

[6]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[7]  Xun Xu,et al.  A Cyber-Physical Machine Tools Platform using OPC UA and MTConnect , 2019, Journal of Manufacturing Systems.

[8]  Michael Heizmann,et al.  INSPECTION OF SPECULAR AND PARTIALLY SPECULAR SURFACES , 2009 .

[9]  Monica Bordegoni,et al.  Towards augmented reality manuals for industry 4.0: A methodology , 2019, Robotics and Computer-Integrated Manufacturing.

[10]  Ben Richard Hughes,et al.  The development of modelling tools to improve energy efficiency in manufacturing processes and systems , 2019, Journal of Manufacturing Systems.

[11]  Xun Xu,et al.  Augmented Reality-assisted Intelligent Window for Cyber-Physical Machine Tools , 2017 .

[12]  Sören Kammel,et al.  Inspection of specular and painted surfaces with centralized fusion techniques , 2006 .

[13]  Pierre Payeur,et al.  Automated surface deformations detection and marking on automotive body panels , 2010, 2010 IEEE International Conference on Automation Science and Engineering.

[14]  Melynda Hoover,et al.  An evaluation of the Microsoft HoloLens for a manufacturing-guided assembly task , 2018 .

[15]  Chase Johnston,et al.  Augmented reality integration of fused LiDAR and spatial mapping , 2018, Commercial + Scientific Sensing and Imaging.

[16]  Roland Hess,et al.  Blender Foundations: The Essential Guide to Learning Blender 2.6 , 2010 .

[17]  Kaj Helin,et al.  User Experience of Augmented Reality System for Astronaut's Manual Work Support , 2018, Front. Robot. AI.

[18]  Eleonora Bottani,et al.  Augmented reality technology in the manufacturing industry: A review of the last decade , 2019, IISE Trans..

[19]  Adrien Coppens,et al.  Merging real and virtual worlds: An analysis of the state of the art and practical evaluation of Microsoft Hololens , 2017, ArXiv.

[20]  Leopoldo Armesto,et al.  Inspection system based on artificial vision for paint defects detection on cars bodies , 2011, 2011 IEEE International Conference on Robotics and Automation.

[21]  Konrad Wegener,et al.  Detection of surface defects on sheet metal parts by using one-shot deflectometry in the infrared range , 2010 .

[22]  Berend Denkena,et al.  Fast Inspection of Larger Sized Curved Surfaces by Stripe Projection , 2002 .

[23]  Simon Jackson.  Unity 3D UI Essentials , 2015 .

[24]  Fumio Kishino,et al.  Augmented reality: a class of displays on the reality-virtuality continuum , 1995, Other Conferences.

[25]  Michael Heizmann,et al.  Infrared deflectometry for the inspection of diffusely specular surfaces , 2016 .

[26]  Roberto Pierdicca,et al.  A Survey of Augmented, Virtual, and Mixed Reality for Cultural Heritage , 2018, ACM Journal on Computing and Cultural Heritage.

[27]  Monica Bordegoni,et al.  Supporting Remote Maintenance in Industry 4.0 through Augmented Reality , 2017 .

[28]  Francisco Simões,et al.  Markerless tracking system for augmented reality in the automotive industry , 2017, Expert Syst. Appl..

[29]  Robin Hanson,et al.  Augmented reality as a means of conveying picking information in kit preparation for mixed-model assembly , 2017, Comput. Ind. Eng..

[30]  Hugh Durrant-Whyte,et al.  Simultaneous localization and mapping (SLAM): part II , 2006 .

[31]  G. L. Cascella,et al.  Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations , 2017, The International Journal of Advanced Manufacturing Technology.

[32]  Jorge Ojeda-Castaneda.  Foucault, Wire, and Phase Modulation Tests , 2006 .

[33]  Josep Tornero,et al.  On the detection of defects on specular car body surfaces , 2017 .

[34]  Josep Tornero,et al.  Detecting dings and dents on specular car body surfaces based on optical flow , 2017 .

[35]  Pierre Payeur,et al.  Features extraction from point clouds for automated detection of deformations on automotive body parts , 2009, 2009 IEEE International Workshop on Robotic and Sensors Environments.

[36]  João Neves,et al.  Application of mixed reality in robot manipulator programming , 2018, Ind. Robot.