Generative Adversarial Networks for Data Augmentation in Structural Adhesive Inspection

The technological advances brought forth by the Industry 4.0 paradigm have renewed the disruptive potential of artificial intelligence in the manufacturing sector, building the data-driven era on top of concepts such as Cyber–Physical Systems and the Internet of Things. However, data availability remains a major challenge for the success of these solutions, particularly concerning those based on deep learning approaches. Specifically in the quality inspection of structural adhesive applications, found commonly in the automotive domain, defect data with sufficient variety, volume and quality is generally costly, time-consuming and inefficient to obtain, jeopardizing the viability of such approaches due to data scarcity. To mitigate this, we propose a novel approach to generate synthetic training data for this application, leveraging recent breakthroughs in training generative adversarial networks with limited data to improve the performance of automated inspection methods based on deep learning, especially for imbalanced datasets. Preliminary results in a real automotive pilot cell show promise in this direction, with the approach being able to generate realistic adhesive bead images and consequently object detection models showing improved mean average precision at different thresholds when trained on the augmented dataset. For reproducibility purposes, the model weights, configurations and data encompassed in this study are made publicly available.

[1]  Frank W. Liou,et al.  Metal Additive Manufacturing Parts Inspection Using Convolutional Neural Network , 2020, Applied Sciences.

[2]  Xu Ke,et al.  Surface defect classification of steels with a new semi-supervised learning method , 2019, Optics and Lasers in Engineering.

[3]  Chien-Yao Wang,et al.  Scaled-YOLOv4: Scaling Cross Stage Partial Network , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Diego Cabrera,et al.  Generative Adversarial Networks Selection Approach for Extremely Imbalanced Fault Diagnosis of Reciprocating Machinery , 2019, IEEE Access.

[5]  Tom White,et al.  Generative Adversarial Networks: An Overview , 2017, IEEE Signal Processing Magazine.

[6]  Jaakko Lehtinen,et al.  GANSpace: Discovering Interpretable GAN Controls , 2020, NeurIPS.

[7]  Fan Yang,et al.  Good Semi-supervised Learning That Requires a Bad GAN , 2017, NIPS.

[8]  Chenglin Wen,et al.  Deep learning fault diagnosis method based on global optimization GAN for unbalanced data , 2020, Knowl. Based Syst..

[9]  Wentao Mao,et al.  Imbalanced Fault Diagnosis of Rolling Bearing Based on Generative Adversarial Network: A Comparative Study , 2019, IEEE Access.

[10]  Jay Lee,et al.  Industrial Artificial Intelligence in Industry 4.0 - Systematic Review, Challenges and Outlook , 2020, IEEE Access.

[11]  Jure Skvarč,et al.  Segmentation-based deep-learning approach for surface-defect detection , 2019, Journal of Intelligent Manufacturing.

[12]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[13]  Abhishek Kumar,et al.  Semi-supervised Learning with GANs: Manifold Invariance with Improved Inference , 2017, NIPS.

[14]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[15]  Li Zhang,et al.  A High-Performance Deep Learning Algorithm for the Automated Optical Inspection of Laser Welding , 2020, Applied Sciences.

[16]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[17]  Tero Karras,et al.  Training Generative Adversarial Networks with Limited Data , 2020, NeurIPS.