Defect Detection and Classification by Training a Generic Convolutional Neural Network Encoder

Non-destructive Testing (NDT) often involves analysing images to identify (rare) defects. We propose a method for locating and classifying abnormalities using Convolutional Neural Networks (CNNs). A particular problem is that it is often difficult to get large numbers of examples of images of defects, making training a classifier challenging. To address this problem we generate large numbers of synthetic images by combining real defects with different backgrounds. These images are used to train a U-Net style network to perform defect detection at the pixel level. We also demonstrate that the encoder of the network produces features which can be applied to the defect classification task at the image level. Both the defect detection and classification modules are tested on multiple small data sets. Our results show that these modules are able to fulfil the industrial component inspection task at the pixel level (locating defect regions) and image level (identifying if an image contains a defect).

[1]  Qingyong Li,et al.  A Hierarchical Extractor-Based Visual Rail Surface Inspection System , 2017, IEEE Sensors Journal.

[2]  Aleksandar Dogandzic,et al.  Bayesian NDE Defect Signal Analysis , 2007, IEEE Transactions on Signal Processing.

[3]  Yonghong Tang,et al.  Automated inspection system for detecting metal surface cracks from fluorescent penetrant images , 1995, Electronic Imaging.

[4]  Jitendra Malik,et al.  Hypercolumns for object segmentation and fine-grained localization , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Hamzah Arof,et al.  Discontinuities detection in welded joints based on inverse surface thresholding , 2011 .

[6]  Jure Skvarč,et al.  Segmentation-based deep-learning approach for surface-defect detection , 2019, Journal of Intelligent Manufacturing.

[7]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[8]  Yibin Huang,et al.  Surface defect saliency of magnetic tile , 2018, The Visual Computer.

[9]  Iasonas Kokkinos,et al.  Deep Filter Banks for Texture Recognition, Description, and Segmentation , 2015, International Journal of Computer Vision.

[10]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Yueming Li,et al.  An automated radiographic NDT system for weld inspection: Part II—Flaw detection , 1998 .

[12]  H. Vincent Poor,et al.  Learning-Based Distributed Detection-Estimation in Sensor Networks With Unknown Sensor Defects , 2017, IEEE Transactions on Signal Processing.

[13]  Gang Wang,et al.  Automatic identification of different types of welding defects in radiographic images , 2002 .

[14]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[15]  Mehryar Mohri,et al.  Confidence Intervals for the Area Under the ROC Curve , 2004, NIPS.

[16]  Walter J. Scheirer,et al.  Neuron Segmentation Using Deep Complete Bipartite Networks , 2017, MICCAI.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Xinghui Dong,et al.  Automatic Inspection of Aerospace Welds Using X-Ray Images , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[19]  Chih-Jen Lin,et al.  Working Set Selection Using Second Order Information for Training Support Vector Machines , 2005, J. Mach. Learn. Res..

[20]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[21]  Joseph Bullock,et al.  XNet: a convolutional neural network (CNN) implementation for medical x-ray image segmentation suitable for small datasets , 2018, Medical Imaging.

[22]  Dina Q. Goldin,et al.  On Similarity Queries for Time-Series Data: Constraint Specification and Implementation , 1995, CP.

[23]  Andrew Zisserman,et al.  A Statistical Approach to Texture Classification from Single Images , 2004, International Journal of Computer Vision.

[24]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[25]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Domingo Mery,et al.  GDXray: The Database of X-ray Images for Nondestructive Testing , 2015, Journal of Nondestructive Evaluation.

[28]  Tania Mezzadri Centeno,et al.  Automated detection of welding defects in pipelines from radiographic images DWDI , 2017 .

[29]  Yuan Yuan,et al.  Research on segmentation and distribution features of small defects in precision weldments with complex structure , 2007 .

[30]  Yunhui Yan,et al.  A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects , 2013 .

[31]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  A. Kehoe,et al.  An intelligent knowledge based approach for the automated radiographic inspection of castings , 1992 .

[33]  Fernand S. Cohen.  Modeling of ultrasound speckle with application in flaw detection in metals , 1992, IEEE Trans. Signal Process..

[34]  Antonio M. López,et al.  The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[36]  Shahram Shahbazpanahi,et al.  A Distributed Reflector Localization Approach to Ultrasonic Array Imaging in Non-Destructive Testing Applications , 2014, IEEE Transactions on Signal Processing.

[37]  Lin Yang,et al.  Deep Adversarial Networks for Biomedical Image Segmentation Utilizing Unannotated Images , 2017, MICCAI.

[38]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[39]  Zaixing He,et al.  Defect detection of castings in radiography images using a robust statistical feature. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[40]  Chihway Chang,et al.  Radio frequency interference mitigation using deep convolutional neural networks , 2016, Astron. Comput..

[41]  Junyu Dong,et al.  Monocular visual-IMU odometry using multi-channel image patch exemplars , 2017, Multimedia Tools and Applications.

[42]  Mohammad R. Jahanshahi,et al.  NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion , 2018, IEEE Transactions on Industrial Electronics.

[43]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[44]  Andrew Zisserman,et al.  A Statistical Approach to Material Classification Using Image Patch Exemplars , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Hristo Bojinov,et al.  Dataset Augmentation with Synthetic Images Improves Semantic Segmentation , 2017, NCVPRIPG.

[46]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[47]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[48]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[49]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[50]  Kay Chen Tan,et al.  A Generic Deep-Learning-Based Approach for Automated Surface Inspection , 2018, IEEE Transactions on Cybernetics.

[51]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Junyu Dong,et al.  The Visual Word Booster: A Spatial Layout of Words Descriptor Exploiting Contour Cues , 2018, IEEE Transactions on Image Processing.

[53]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[54]  Hao Wang,et al.  Detection of line weld defects based on multiple thresholds and support vector machine , 2008 .

[55]  Xinghui Dong,et al.  Small Defect Detection Using Convolutional Neural Network Features and Random Forests , 2018, ECCV Workshops.