Automatic Visual Inspection of Rare Defects: A Framework based on GP-WGAN and Enhanced Faster R-CNN

A current trend in industries such as semiconductors and foundry is to shift their visual inspection processes to Automatic Visual Inspection (AVI) systems, to reduce their costs, mistakes, and dependency on human experts. This paper proposes a two-staged fault diagnosis framework for AVI systems. In the first stage, a generation model is designed to synthesize new samples based on real samples. The proposed augmentation algorithm extracts objects from the real samples and blends them randomly, to generate new samples and enhance the performance of the image processor. In the second stage, an improved deep learning architecture based on Faster R-CNN, Feature Pyramid Network (FPN), and a Residual Network is proposed to perform object detection on the enhanced dataset. The performance of the algorithm is validated and evaluated on two multi-class datasets. The experimental results performed over a range of imbalance severities demonstrate the superiority of the proposed framework compared to other solutions.

[1]  Nickolas M. Wergeles,et al.  A survey and performance evaluation of deep learning methods for small object detection , 2021, Expert Syst. Appl..

[2]  J. Gómez-Sanchís,et al.  Advances in Machine Vision Applications for Automatic Inspection and Quality Evaluation of Fruits and Vegetables , 2011 .

[3]  S. R. Aghdam,et al.  A fast method of steel surface defect detection using decision trees applied to LBP based features , 2012, 2012 7th IEEE Conference on Industrial Electronics and Applications (ICIEA).

[4]  Yibin Huang,et al.  Surface defect saliency of magnetic tile , 2018, The Visual Computer.

[5]  Safwan Ahmad,et al.  Automated detection of textured-surface defects using UNet-based semantic segmentation network , 2020, 2020 IEEE International Conference on Prognostics and Health Management (ICPHM).

[6]  W-Y Wu,et al.  Automated metal surface inspection through machine vision , 2002 .

[7]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[8]  Reinhold Huber-Mörk,et al.  Convolutional Neural Networks for Steel Surface Defect Detection from Photometric Stereo Images , 2014, ISVC.

[9]  Franz Pernkopf,et al.  Detection of surface defects on raw steel blocks using Bayesian network classifiers , 2004, Pattern Analysis and Applications.

[10]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[11]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[12]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[13]  Yu He,et al.  Unified detection method of aluminium profile surface defects: Common and rare defect categories , 2020 .

[14]  Du-Ming Tsai,et al.  Morphology-based defect detection in machined surfaces with circular tool-mark patterns , 2019, Measurement.

[15]  Grantham Pang,et al.  Fabric inspection based on the Elo rating method , 2016, Pattern Recognit..

[16]  Du-Ming Tsai,et al.  Autoencoder-based anomaly detection for surface defect inspection , 2021, Adv. Eng. Informatics.

[17]  Qinggang Meng,et al.  Unsupervised Saliency Detection of Rail Surface Defects Using Stereoscopic Images , 2021, IEEE Transactions on Industrial Informatics.

[18]  Yanjie Lu,et al.  An intelligent machine vision system for detecting surface defects on packing boxes based on support vector machine , 2019, Measurement and Control.

[19]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Bernd Scholz-Reiter,et al.  Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection , 2016 .

[21]  Hui Lin,et al.  Defect Image Sample Generation With GAN for Improving Defect Recognition , 2020, IEEE Transactions on Automation Science and Engineering.

[22]  Jianhui Wang,et al.  Detection of PCB Surface Defects With Improved Faster-RCNN and Feature Pyramid Network , 2020, IEEE Access.

[23]  Yuanxiang Li,et al.  A Faster-RCNN Based Chemical Fiber Paper Tube Defect Detection Method , 2017, 2017 5th International Conference on Enterprise Systems (ES).

[24]  Ying Cheng,et al.  A steel surface defect inspection approach towards smart industrial monitoring , 2020, Journal of Intelligent Manufacturing.

[25]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[26]  Quoc V. Le,et al.  NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Bachir Boudraa,et al.  Mathematical morphology for TOFD image analysis and automatic crack detection. , 2014, Ultrasonics.

[28]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Carlo Vercellis,et al.  Fault detection and diagnosis for rotating machinery: A model based on convolutional LSTM, Fast Fourier and continuous wavelet transforms , 2021, Comput. Ind..

[30]  Chen Jie,et al.  A deep learning-based approach for the automated surface inspection of copper clad laminate images , 2020 .

[31]  J. de Brito,et al.  Inspection and Diagnosis System for Wood Flooring , 2013 .

[32]  Sun Min,et al.  Double Feature Extraction Method for Wafer Map Classification Based on Convolution Neural Network , 2020, 2020 31st Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC).