Vision guided robotic inspection for parts in manufacturing and remanufacturing industry

Environmental and commercial drivers are leading to a circular economy where systems and components are routinely recycled or remanu- factured. Unlike traditional manufacturing, where components typically have a high degree of tolerance, components in the remanufacturing process may have seen decades of wear, resulting in a wider variation of geometries. This makes it difficult to translate existing automation techniques to perform Non- Destructive Testing (NDT) for such components autonomously. The challenge of performing automated inspections, with off-line tool-paths developed from Computer Aided Design (CAD) models, typically arises from the fact that those paths do not have the required level of accuracy. Beside the fact that CAD models are less available for old parts, these parts often differ from their respective virtual models. This paper considers exible automation by com- bining part geometry reconstruction with ultrasonic tool-path generation, to perform Ultrasonic NDT. This paper presents an approach to perform custom vision-guided ultrasonic inspection of components, which is achieved through integrating an automated vision system and a purposely developed graphic user interface with a robotic work-cell. The vision system, based on structure from motion, allows creating 3D models of the parts. Also, this work compares four dierent tool-paths for optimum image capture. The resulting optimum 3D models are used in a virtual twin environment of the robotic inspection cell, to enable the user to select any points of interest for ultrasonic inspection. This removes the need of offine robot path-planning and part orientation for assessing specic locations on a part, which is typically a very time-consuming phase.

[1]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[3]  Richard French,et al.  Intelligent sensing for robotic re-manufacturing in aerospace — An industry 4.0 design based prototype , 2017, 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS).

[4]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[5]  Sei Ikeda,et al.  Visual SLAM algorithms: a survey from 2010 to 2016 , 2017, IPSJ Transactions on Computer Vision and Applications.

[6]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[7]  Charles Norman Macleod,et al.  Machining-Based Coverage Path Planning for Automated Structural Inspection , 2018, IEEE Transactions on Automation Science and Engineering.

[8]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[9]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[10]  Paolo Cignoni,et al.  MeshLab: an Open-Source Mesh Processing Tool , 2008, Eurographics Italian Chapter Conference.

[11]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[12]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[13]  Alessandra Caggiano,et al.  Digital factory technologies for robotic automation and enhanced manufacturing cell design , 2018 .

[14]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[16]  Dennis J Thevara,et al.  Application of photogrammetry to automated finishing operations , 2018 .

[17]  Carmelo Mineo,et al.  PAUT inspection of complex shaped composite materials through 6 DOFs robotic manipulators , 2015 .

[18]  Pia Benaud,et al.  Two decades of digital photogrammetry: Revisiting Chandler’s 1999 paper on “Effective application of automated digital photogrammetry for geomorphological research” – a synthesis , 2019, Progress in Physical Geography: Earth and Environment.

[19]  Carmelo Mineo,et al.  Robotic path planning for non-destructive testing – A custom MATLAB toolbox approach , 2016 .

[20]  Jianhua Liu,et al.  A 3D reconstruction method for pipeline inspection based on multi-vision , 2017 .

[21]  Andrew J. Davison,et al.  Real-time simultaneous localisation and mapping with a single camera , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[22]  Antonios Gasteratos,et al.  Multi-camera 3D Object Reconstruction for Industrial Automation , 2012, APMS.

[23]  Aasim Khurshid,et al.  A Robust and Real-Time Face Anti-spoofing Method Based on Texture Feature Analysis , 2019, HCI.

[24]  Iliyana Samardzhieva,et al.  Neccessasity of Bio-imaging Hybrid Approaches Accelerating Drug Discovery Process (Mini-Review) , 2018, International Journal of Computer Applications.

[25]  J. Valença,et al.  Applications of photogrammetry to structural assessment , 2012, Experimental Techniques.

[26]  Gordon Dobie,et al.  Quantifying impacts on remote photogrammetric inspection using unmanned aerial vehicles , 2020, Engineering Structures.

[27]  Erfu Yang,et al.  Interfacing Toolbox for Robotic Arms with Real-Time Adaptive Behavior Capabilities , 2019 .

[28]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[29]  Rahul Summan,et al.  Flexible integration of robotics, ultrasonics and metrology for the inspection of aerospace components , 2017 .

[30]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[31]  Antonios Gasteratos,et al.  Multi-camera 3D scene reconstruction from vanishing points , 2010, 2010 IEEE International Conference on Imaging Systems and Techniques.

[32]  Qi Peng,et al.  Automated 3D Scenes Reconstruction Using Multiple Stereo Pairs from Portable Four-Camera Photographic Measurement System , 2015 .

[33]  Jacob Scharcanski,et al.  An Adaptive Face Tracker with Application in Yawning Detection , 2020, Sensors.

[34]  Gordon Dobie,et al.  A Feasibility Study on Guided Wave- Based Robotic Mapping , 2019, 2019 IEEE International Ultrasonics Symposium (IUS).

[35]  Carmelo Mineo,et al.  Enabling robotic adaptive behaviour capabilities for new Industry 4.0 automated quality inspection paradigms , 2020, Insight - Non-Destructive Testing and Condition Monitoring.