Automatic Detection and Location of Weld Beads With Deep Convolutional Neural Networks

Welding quality detection is a critical link in modern manufacturing, and the weld bead location is a prerequisite for the high-precision assessment of welding quality. It is generally necessary for weld bead detection to be accomplished in the context of complex industrial environments. However, conventional detection and location methods based on specific detection conditions or prior knowledge lack accuracy and adaptability. To precisely detect and locate the weld beads in real industrial environments, a novel weld bead detection and location algorithm is proposed based on deep convolutional neural networks. Because there is no open data set of weld beads and the samples in real industrial applications are insufficient for effective model training of the deep convolutional neural network, a novel data augmentation method based on a deep semantic segmentation network is proposed to increase the sample diversity and enlarge the data set. Then, a dynamic sample updating strategy is put forward to cover more welding situations. Finally, faced with the weak-feature and weak-texture characteristics of weld beads, a simplified YOLOV3 model is proposed to realize end-to-end weld bead location. Experiments demonstrate that the proposed method could effectively satisfy the robustness and precision requirements for weld bead detection and location combined with a deep semantic segmentation network and simplified YOLOV3 model.

[1]  Giuseppe Casalino,et al.  Investigation on Ti6Al4V laser welding using statistical and Taguchi approaches , 2005 .

[2]  U. Natarajan,et al.  Vision inspection system for the identification and classification of defects in MIG welding joints , 2012 .

[3]  Xian Tao,et al.  Detection of Power Line Insulator Defects Using Aerial Images Analyzed With Convolutional Neural Networks , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[4]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Harry A. Pierson,et al.  Deep learning in robotics: a review of recent research , 2017, Adv. Robotics.

[6]  Zong-Yi Wang,et al.  A vision-based system for post-welding quality measurement and defect detection , 2016 .

[7]  A. Azari Moghaddam,et al.  Classification of welding defects in radiographic images , 2015, Pattern Recognition and Image Analysis.

[8]  Ninshu Ma,et al.  Measurement of residual stress in arc welded lap joints by cosα X-ray diffraction method , 2017 .

[9]  Deyong You,et al.  WPD-PCA-Based Laser Welding Process Monitoring and Defects Diagnosis by Using FNN and SVM , 2015, IEEE Transactions on Industrial Electronics.

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Guangrui Wen,et al.  Random forest-based real-time defect detection of Al alloy in robotic arc welding using optical spectrum , 2019, Journal of Manufacturing Processes.

[12]  Lei Yang,et al.  A precise seam tracking method for narrow butt seams based on structured light vision sensor , 2019, Optics & Laser Technology.

[13]  Dong Xu,et al.  Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey , 2018, IEEE Signal Processing Magazine.

[14]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Lei Yang,et al.  An Automatic Detection and Identification Method of Welded Joints Based on Deep Neural Network , 2019, IEEE Access.

[16]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[17]  S. Katayama,et al.  A Novel Stability Quantification for Disk Laser Welding by Using Frequency Correlation Coefficient Between Multiple-Optics Signals , 2015, IEEE/ASME Transactions on Mechatronics.

[18]  Hao Zeng,et al.  Application of artificial neural network in laser welding defect diagnosis , 2005 .

[19]  Shuzhi Sam Ge,et al.  Automatic Welding Seam Tracking and Identification , 2017, IEEE Transactions on Industrial Electronics.

[20]  Guangrui Wen,et al.  Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding , 2019, Journal of Manufacturing Processes.

[21]  De Xu,et al.  A Vision-Based Self-Tuning Fuzzy Controller for Fillet Weld Seam Tracking , 2011, IEEE/ASME Transactions on Mechatronics.

[22]  Yu-Kang Liu,et al.  Supervised Learning of Human Welder Behaviors for Intelligent Robotic Welding , 2017, IEEE Transactions on Automation Science and Engineering.

[23]  Andrés Montoyo,et al.  Advances on natural language processing , 2007, Data Knowl. Eng..

[24]  Jinna Qin,et al.  Real-Time Trajectory Compensation in Robotic Friction Stir Welding Using State Estimators , 2016, IEEE Transactions on Control Systems Technology.

[25]  Lei Yang,et al.  A Novel 3-D Path Extraction Method for Arc Welding Robot Based on Stereo Structured Light Sensor , 2019, IEEE Sensors Journal.

[26]  Sheng Liu,et al.  Welding quality monitoring of high frequency straight seam pipe based on image feature , 2017 .

[27]  Lei Yang,et al.  A welding quality detection method for arc welding robot based on 3D reconstruction with SFS algorithm , 2018 .

[28]  Thomas R. Kurfess,et al.  Online quality inspection using Bayesian classification in powder-bed additive manufacturing from high-resolution visual camera images , 2018, Journal of Intelligent Manufacturing.

[29]  Geok Soon Hong,et al.  Defect detection in selective laser melting technology by acoustic signals with deep belief networks , 2018, The International Journal of Advanced Manufacturing Technology.

[30]  De Xu,et al.  Measurement and Defect Detection of the Weld Bead Based on Online Vision Inspection , 2010, IEEE Transactions on Instrumentation and Measurement.

[31]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[32]  Steven Delrue,et al.  Non-destructive ultrasonic examination of root defects in friction stir welded butt-joints , 2016 .

[33]  Gil-Sang Yoon,et al.  A feature-based inspection planning system for coordinate measuring machines , 2005 .

[34]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Yu Yao,et al.  Detection of a casting defect tracked by deep convolution neural network , 2018, The International Journal of Advanced Manufacturing Technology.

[36]  Rama Chellappa,et al.  HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Wei Yang,et al.  Weld Line Detection and Tracking via Spatial-Temporal Cascaded Hidden Markov Models and Cross Structured Light , 2014, IEEE Transactions on Instrumentation and Measurement.

[38]  YuMing Zhang,et al.  Iterative Local ANFIS-Based Human Welder Intelligence Modeling and Control in Pipe GTAW Process: A Data-Driven Approach , 2015, IEEE/ASME Transactions on Mechatronics.

[39]  Tania Mezzadri Centeno,et al.  Automated detection of welding defects in pipelines from radiographic images DWDI , 2017 .

[40]  Yongji Wang,et al.  A 3D reconstruction method based on grid laser and gray scale photo for visual inspection of welds , 2019, Optics & Laser Technology.

[41]  B. Li,et al.  Simultaneous Monitoring of Penetration Status and Joint Tracking During Laser Keyhole Welding , 2019, IEEE/ASME Transactions on Mechatronics.

[42]  Lin Tao,et al.  Inspection of weld shape based on the shape from shading , 2006 .

[43]  Patrick M. Pilarski,et al.  Intelligent laser welding through representation, prediction, and control learning: An architecture with deep neural networks and reinforcement learning , 2016 .