Real-Time High-Performance Laser Welding Defect Detection by Combining ACGAN-Based Data Enhancement and Multi-Model Fusion

Most of the existing laser welding process monitoring technologies focus on the detection of post-engineering defects, but in the mass production of electronic equipment, such as laser welding metal plates, the real-time identification of defect detection has more important practical significance. The data set of laser welding process is often difficult to build and there is not enough experimental data, which hinder the applications of the data-driven laser welding defect detection method. In this paper, an intelligent welding defect diagnosis method based on auxiliary classifier generative adversarial networks (ACGAN) has been proposed. Firstly, a ten-class dataset consisting of 6467 samples, was constructed, which originate from the optical and thermal sensory parameters in the welding process. A new structured ACGAN network model is proposed to generate fake data similar to the true defect feature distributions. In addition, in order to make the difference between different defects categories more obvious after data expansion, a data filtering and data purification scheme was proposed based on ensemble learning and an SVM (support vector machine), which is used to filter the bad generated data. In the experiments, the classification accuracy can reach 96.83% and 85.13%, for the CNN (convolutional neural network) algorithm model and ACGAN model, respectively. However, the accuracy can further improve to 97.86% and 98.37% for the fusion models of ACGAN-CNN and ACGAN-SVM-CNN models, respectively. The results show that ACGAN can not only be used as an algorithm model for classification, but also be used to achieve superior real-time classification and recognition through data enhancement and multi-model fusion.

[1]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[2]  Sergey A. Shevchik,et al.  Laser Welding Quality Monitoring via Graph Support Vector Machine With Data Adaptive Kernel , 2019, IEEE Access.

[3]  S. Katayama,et al.  Review of laser welding monitoring , 2014 .

[4]  Bin Zhang,et al.  Multidimensional Feature Fusion and Ensemble Learning-Based Fault Diagnosis for the Braking System of Heavy-Haul Train , 2021, IEEE Transactions on Industrial Informatics.

[5]  He Zhang,et al.  PWM-VSI Fault Diagnosis for a PMSM Drive Based on the Fuzzy Logic Approach , 2019, IEEE Transactions on Power Electronics.

[6]  Antti Salminen,et al.  Monitoring and Adaptive Control of Laser Processes , 2014 .

[7]  Zhiqiang Ge,et al.  Data Augmentation Classifier for Imbalanced Fault Classification , 2021, IEEE Transactions on Automation Science and Engineering.

[8]  M. Hertel,et al.  Thermal Efficiency Analysis for Laser-Assisted Plasma Arc Welding of AISI 304 Stainless Steel , 2019, Materials.

[9]  Jieyun Bai,et al.  Cardiotocography signal abnormality classification using time-frequency features and Ensemble Cost-sensitive SVM classifier , 2021, Comput. Biol. Medicine.

[10]  A. Krajewski,et al.  A Novel Method of Supporting the Laser Welding Process with Mechanical Acoustic Vibrations , 2020, Materials.

[11]  Young Whan Park,et al.  Real time estimation of CO2 laser weld quality for automotive industry , 2002 .

[12]  Geoffrey I. Webb,et al.  Multistrategy ensemble learning: reducing error by combining ensemble learning techniques , 2004, IEEE Transactions on Knowledge and Data Engineering.

[13]  Sandeep Kumar,et al.  WR-ELM: Weighted Regularization Extreme Learning Machine for Imbalance Learning in Software Fault Prediction , 2020, IEEE Transactions on Reliability.

[14]  Radovan Kovacevic,et al.  A Laser-Based Vision System for Weld Quality Inspection , 2011, Sensors.

[15]  Hong Chen,et al.  Unidimensional ACGAN Applied to Link Establishment Behaviors Recognition of a Short-Wave Radio Station , 2020, Sensors.

[16]  Jae Sang Moon,et al.  Development of a Seismic Detection Technology for High-Speed Trains Using Signal Analysis Techniques , 2020, Sensors.

[17]  Wei Cai,et al.  A Lighted Deep Convolutional Neural Network Based Fault Diagnosis of Rotating Machinery , 2019, Sensors.

[18]  Yanxi Zhang,et al.  Data-Driven Detection of Laser Welding Defects Based on Real-Time Spectrometer Signals , 2019, IEEE Sensors Journal.

[19]  Dan Wang,et al.  Detection and Diagnosis of Defect in GIS Based on X-ray Digital Imaging Technology , 2020 .

[20]  Hsiang‐Chen Chui,et al.  Visual-assisted laser microwelding of carbon microfiber on metal plates , 2018, Optics & Laser Technology.

[21]  M. Landowski,et al.  Autogenous Fiber Laser Welding of 316L Austenitic and 2304 Lean Duplex Stainless Steels , 2020, Materials.

[22]  D. Rodrigues,et al.  Analysis of weld defects in similar and dissimilar resistance seam welding of aluminium, zinc and galvanised steel , 2015 .

[23]  Yanxi Zhang,et al.  Online Monitoring of Welding Status Based on a DBN Model During Laser Welding , 2019, Engineering.

[24]  Junfeng Jing,et al.  Pixel-Wise Fabric Defect Detection by CNNs Without Labeled Training Data , 2020, IEEE Access.

[25]  T. Gang,et al.  Intelligent pattern recognition and diagnosis of ultrasonic inspection of welding defects based on neural network and information fusion , 2002 .

[26]  Yang Hu,et al.  Study of joint temporal-spatial distribution of array output for large-scale photovoltaic plant and its fault diagnosis application , 2019 .

[27]  Jiong Tang,et al.  Preprocessing-Free Gear Fault Diagnosis Using Small Datasets With Deep Convolutional Neural Network-Based Transfer Learning , 2017, IEEE Access.

[28]  Li Tian,et al.  Fault Diagnostic Method Based on Deep Learning and Multimodel Feature Fusion for Complex Industrial Processes , 2020 .

[29]  Youren Wang,et al.  Fault diagnosis of planetary gearbox using multi-criteria feature selection and heterogeneous ensemble learning classification , 2020 .

[30]  Ke Zhang,et al.  Detection Method Based on Automatic Visual Shape Clustering for Pin-Missing Defect in Transmission Lines , 2020, IEEE Transactions on Instrumentation and Measurement.

[31]  Sai Ho Ling,et al.  A Hybrid Feature Selection and Extraction Methods for Sleep Apnea Detection Using Bio-Signals , 2020, Sensors.

[32]  Youmin Zhang,et al.  Weld Defect Detection From Imbalanced Radiographic Images Based on Contrast Enhancement Conditional Generative Adversarial Network and Transfer Learning , 2021, IEEE Sensors Journal.

[33]  Yu Huang,et al.  Automatic weld seam tracking of tube-to-tubesheet TIG welding robot with multiple sensors , 2020 .

[34]  Kai Keng Ang,et al.  Generative Adversarial Networks-Based Data Augmentation for Brain–Computer Interface , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[35]  K. Pańcikiewicz,et al.  Laser Dissimilar Welding of AISI 430F and AISI 304 Stainless Steels , 2020, Materials.

[36]  Yingying Chen,et al.  A Multi-Sensor Data Fusion System for Laser Welding Process Monitoring , 2020, IEEE Access.

[37]  Fred A. Hamprecht,et al.  Sputter Tracking for the Automatic Monitoring of Industrial Laser-Welding Processes , 2008, IEEE Transactions on Industrial Electronics.

[38]  M. Sahul,et al.  Effect of Disk Laser Beam Offset on the Microstructure and Mechanical Properties of Copper—AISI 304 Stainless Steel Dissimilar Metals Joints , 2020 .

[39]  Witold Pedrycz,et al.  A Study on Relationship Between Generalization Abilities and Fuzziness of Base Classifiers in Ensemble Learning , 2015, IEEE Transactions on Fuzzy Systems.

[40]  Dong Du,et al.  In-Process Monitoring of Lack of Fusion in Ultra-Thin Sheets Edge Welding Using Machine Vision , 2018, Sensors.

[41]  Perry P. Gao,et al.  Welding defects detection based on deep learning with multiple optical sensors during disk laser welding of thick plates , 2019, Journal of Manufacturing Systems.

[42]  Yaonan Wang,et al.  A Surface Defect Detection Framework for Glass Bottle Bottom Using Visual Attention Model and Wavelet Transform , 2020, IEEE Transactions on Industrial Informatics.

[43]  Rui Yang,et al.  An Improved Fault Diagnosis Method of Rotating Machinery Using Sensitive Features and RLS-BP Neural Network , 2020, IEEE Transactions on Instrumentation and Measurement.

[44]  Wensu Chen,et al.  Near real‐time bolt‐loosening detection using mask and region‐based convolutional neural network , 2021, Structural Control and Health Monitoring.

[45]  Sadek C. A. Alfaro,et al.  A Weld Defects Detection System Based on a Spectrometer , 2009, Sensors.

[46]  Deepak Gupta,et al.  CovidGAN: Data Augmentation Using Auxiliary Classifier GAN for Improved Covid-19 Detection , 2020, IEEE Access.

[47]  D. Faucher,et al.  High-Power and Widely Tunable All-Fiber Raman Laser , 2008, Journal of Lightwave Technology.

[48]  Adolfo Cobo,et al.  Closed-loop power and focus control of laser welding for full-penetration monitoring. , 2005, Applied optics.