Automatic Identification of Multi-Type Weld Seam Based on Vision Sensor With Silhouette-Mapping

Automatic identification of weld seam types by welding robot is a key link in intelligent welding as some adjustment scheme (e.g., welding trajectory planning, initial welding position, welding parameters) vary with the weld seam types. However, the variable welding environment and various weld seam profile omnifarious affect the robustness of weld seam types identification. To overcome the challenges derived from the weld seam diversity, in this paper, the silhouette-mapping was selected as the weld seam intermedium and a multi-type weld seam automatic identification system based on vision sensor was introduced. Two different laser sources were adopted to obtain robust silhouette-mapping features in proper gestures. Based on the silhouette-mapping data (stripe-mapping and spot-mapping), the related image processing algorithms were carried out to achieve automatic identification. Specifically, the bidirectional deviation search method was proposed to locate the spot-mapping area based on the stripe-mapping image. Aiming at the characteristics of the spot-mapping image, a carefully designed CNN (convolutional neural network) model was used to classify types. Experimental results prove that the silhouette-mapping and CNN are an effective combination for the multi-type weld seam identification, and a total of 97.6% of weld seam types were correctly predicted. Some weld-related studies include welding features extraction, and welding quality detection may improve its accuracy on the basis of determining weld seam types.

[1]  Jian Chen,et al.  A robust method to extract a laser stripe centre based on grey level moment , 2015 .

[2]  Daniel F. García,et al.  Fast and robust laser stripe extraction for 3D reconstruction in industrial environments , 2010, Machine Vision and Applications.

[3]  Marizan Sulaiman,et al.  Butt welding joints recognition and location identification by using local thresholding , 2018, Robotics and Computer-Integrated Manufacturing.

[4]  Nianfeng Wang,et al.  A robust weld seam recognition method under heavy noise based on structured-light vision , 2020, Robotics Comput. Integr. Manuf..

[5]  Yanling Xu,et al.  An adaptive feature extraction algorithm for multiple typical seam tracking based on vision sensor in robotic arc welding , 2019, Sensors and Actuators A: Physical.

[6]  Wen Wan,et al.  Autonomous seam acquisition and tracking system for multi-pass welding based on vision sensor , 2013 .

[7]  Tao Lin,et al.  Efficient weld seam detection for robotic welding based on local image processing , 2009, Ind. Robot.

[8]  Radovan Kovacevic,et al.  Development of a real-time laser-based machine vision system to monitor and control welding processes , 2012 .

[9]  Y. Shin,et al.  Deep-learning-based porosity monitoring of laser welding process , 2020 .

[10]  Jie Zhang,et al.  Online defect recognition of narrow overlap weld based on two-stage recognition model combining continuous wavelet transform and convolutional neural network , 2019, Comput. Ind..

[11]  Gu Fang,et al.  Detection of fillet weld joints using an adaptive line growing algorithm for robotic arc welding , 2014 .

[12]  Rudolf Weber,et al.  Position sensing of ultrashort pulsed laser-welded seams in glass by optical coherence tomography , 2020 .

[13]  Wei Liu,et al.  Automatic Detection and Segmentation of Laser Stripes for Industrial Measurement , 2020, IEEE Transactions on Instrumentation and Measurement.

[14]  Fan Yang,et al.  Stainless Steel Weld Defect Detection Using Pulsed Inductive Thermography , 2016, IEEE Transactions on Applied Superconductivity.

[15]  Jeehyun Kim,et al.  Non-Destructive Identification of Weld-Boundary and Porosity Formation During Laser Transmission Welding by Using Optical Coherence Tomography , 2018, IEEE Access.

[16]  Shuzhi Sam Ge,et al.  Automatic Welding Seam Tracking and Identification , 2017, IEEE Transactions on Industrial Electronics.

[17]  Gu Fang,et al.  Autonomous weld seam identification and localisation using eye-in-hand stereo vision for robotic arc welding , 2013 .

[18]  Sang Bong Kim,et al.  Development of high speed rotating arc sensor and seam tracking controller for welding robots , 2001, ISIE 2001. 2001 IEEE International Symposium on Industrial Electronics Proceedings (Cat. No.01TH8570).

[19]  Carlo Alberto Avizzano,et al.  A Smart Monitoring System for Automatic Welding Defect Detection , 2019, IEEE Transactions on Industrial Electronics.

[20]  Robert Schmitt,et al.  3D Polymer Weld Seam Characterization Based on Optical Coherence Tomography for Laser Transmission Welding Applications , 2014 .

[21]  Lining Sun,et al.  Weld seam detection based on visual saliency for autonomous welding robots , 2016, 2016 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO).

[22]  Lei Yang,et al.  A welding quality detection method for arc welding robot based on 3D reconstruction with SFS algorithm , 2018 .

[23]  Zong-Yi Wang,et al.  Precise initial weld position identification of a fillet weld seam using laser vision technology , 2018, The International Journal of Advanced Manufacturing Technology.

[24]  Weifeng Zhang,et al.  Real-time penetration state monitoring using convolutional neural network for laser welding of tailor rolled blanks , 2020 .

[25]  Michael F. Zaeh,et al.  Inline weld depth measurement for high brilliance laser beam sources using optical coherence tomography , 2019, Journal of Laser Applications.

[26]  Jürgen Schmidhuber,et al.  Steel defect classification with Max-Pooling Convolutional Neural Networks , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[27]  Kang-Yul Bae,et al.  A study on development of inductive sensor for automatic weld seam tracking , 2006 .

[28]  Lei Yang,et al.  A Precise Initial Weld Point Guiding Method of Micro-Gap Weld Based on Structured Light Vision Sensor , 2019, IEEE Sensors Journal.

[29]  Yongji Wang,et al.  A 3D reconstruction method based on grid laser and gray scale photo for visual inspection of welds , 2019, Optics & Laser Technology.

[30]  Min Tan,et al.  Automatic recognition system of welding seam type based on SVM method , 2017, The International Journal of Advanced Manufacturing Technology.

[31]  Radovan Kovacevic,et al.  An on-line shape-matching weld seam tracking system , 2016 .

[32]  Shuzhi Sam Ge,et al.  Robust Welding Seam Tracking and Recognition , 2017, IEEE Sensors Journal.

[33]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[34]  Meng Shi,et al.  Magnetic flux leakage image classification method for pipeline weld based on optimized convolution kernel , 2019, Neurocomputing.

[35]  Lijian Yang,et al.  Pipeline Magnetic Flux Leakage Image Detection Algorithm Based on Multiscale SSD Network , 2020, IEEE Transactions on Industrial Informatics.

[36]  Yudong Wang,et al.  Feature Extraction of Welding Seam Image Based on Laser Vision , 2018, IEEE Sensors Journal.

[37]  Guangrui Wen,et al.  Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding , 2019, Journal of Manufacturing Processes.

[38]  Lei Yang,et al.  A Novel 3-D Path Extraction Method for Arc Welding Robot Based on Stereo Structured Light Sensor , 2019, IEEE Sensors Journal.