Real-time assistance to manual assembly through depth camera and visual feedback

Abstract The current fourth industrial revolution significantly impacts on production processes. The personalized production paradigm enables customers to order unique products. The operators assemble an enormous component variety adapting their process from product to product with limited learning opportunities. Digital technologies are increasingly adopted in production processes to improve performance and quality. Considering this framework, this research proposes a hardware/software architecture to assist in real-time operators involved in manual assembly processes. A depth camera captures human motions in relation with the workstation environment whereas a visual feedback guides the operator through consecutive assembly tasks. An industrial case study validates the architecture.

[1]  B Bonnechère,et al.  Determination of the precision and accuracy of morphological measurements using the Kinect™ sensor: comparison with standard stereophotogrammetry , 2014, Ergonomics.

[2]  Johan Stahre,et al.  Cognitive automation strategy for reconfigurable and sustainable assembly systems , 2013 .

[3]  Yue Wang,et al.  Point cloud and visual feature-based tracking method for an augmented reality-aided mechanical assembly system , 2018, The International Journal of Advanced Manufacturing Technology.

[4]  Marco Bortolini,et al.  Reconfigurable manufacturing systems: Literature review and research trend , 2018, Journal of Manufacturing Systems.

[5]  Sen Yang,et al.  Mechanical assembly assistance using marker-less augmented reality system , 2018 .

[6]  Gabriele Bleser,et al.  Innovative system for real-time ergonomic feedback in industrial manufacturing. , 2013, Applied ergonomics.

[7]  Antonija Mitrovic,et al.  Intelligent Augmented Reality Training for Motherboard Assembly , 2015, International Journal of Artificial Intelligence in Education.

[8]  Mohamed Khamis,et al.  Introduction and establishment of virtual training in the factory of the future , 2017, Int. J. Comput. Integr. Manuf..

[9]  Ciro A. Rodríguez,et al.  Next-generation manufacturing systems: key research issues in developing and integrating reconfigurable and intelligent machines , 2005, Int. J. Comput. Integr. Manuf..

[10]  Dominic Gorecky,et al.  Developing a Mixed Reality Assistance System Based on Projection Mapping Technology for Manual Operations at Assembly Workstations , 2015 .

[11]  Mauro Gamberi,et al.  Including Material Exposure and Part Attributes in the Manual Assembly Line Balancing Problem , 2016 .

[12]  Raymond W. McGorry,et al.  The validity of the first and second generation Microsoft Kinect™ for identifying joint center locations during static postures. , 2015, Applied ergonomics.

[13]  Rafael Radkowski,et al.  Object Tracking With a Range Camera for Augmented Reality Assembly Assistance , 2016, J. Comput. Inf. Sci. Eng..

[14]  Florian Geiselhart,et al.  On the Use of Multi-Depth-Camera Based Motion Tracking Systems in Production Planning Environments , 2016 .

[15]  Enrico Rukzio,et al.  Dual Reality for Production Verification Workshops: A Comprehensive Set of Virtual Methods , 2016 .

[16]  Franco Failli,et al.  An Integrated Environment Based on Augmented Reality and Sensing Device for Manual Assembly Workstations , 2016 .