A deep-learning-based approach for fast and robust steel surface defects classification

Abstract Automatic visual recognition of steel surface defects provides critical functionality to facilitate quality control of steel strip production. In this paper, we present a compact yet effective convolutional neural network (CNN) model, which emphasizes the training of low-level features and incorporates multiple receptive fields, to achieve fast and accurate steel surface defect classification. Our proposed method adopts the pre-trained SqueezeNet as the backbone architecture. It only requires a small amount of defect-specific training samples to achieve high-accuracy recognition on a diversity-enhanced testing dataset of steel surface defects which contains severe non-uniform illumination, camera noise, and motion blur. Moreover, our proposed light-weight CNN model can meet the requirement of real-time online inspection, running over 100 fps on a computer equipped with a single NVIDIA TITAN X Graphics Processing Unit (12G memory). Codes and a diversity-enhanced testing dataset will be made publicly available.

[1]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[3]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Yi Lu Murphey,et al.  An intelligent real-time vision system for surface defect detection , 2004, ICPR 2004.

[5]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[6]  Yunhui Yan,et al.  A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects , 2013 .

[7]  Sangchul Won,et al.  Vision-based inspection for periodic defects in steel wire rod production , 2010 .

[8]  Anirban Mukherjee,et al.  Automatic Defect Detection on Hot-Rolled Flat Steel Products , 2013, IEEE Transactions on Instrumentation and Measurement.

[9]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[10]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jean-Luc Dugelay,et al.  Learned vs. Hand-Crafted Features for Pedestrian Gender Recognition , 2015, ACM Multimedia.

[12]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[13]  Reinhold Huber-Mörk,et al.  Convolutional Neural Networks for Steel Surface Defect Detection from Photometric Stereo Images , 2014, ISVC.

[14]  Sang Woo Kim,et al.  Defect detection for corner cracks in steel billets using a wavelet reconstruction method. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[15]  Li Yi,et al.  An End‐to‐End Steel Strip Surface Defects Recognition System Based on Convolutional Neural Networks , 2017 .

[16]  Yu Xie,et al.  A physics-based defects model and inspection algorithm for automatic visual inspection , 2014 .

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[19]  J.S.Chitode,et al.  Metal Surface Inspection for Defect Detectionand Classification using Gabor Filter , 2014 .

[20]  Roman V. Yampolskiy,et al.  Adaptive Extended Local Ternary Pattern (AELTP) for Recognizing Avatar Faces , 2012, 2012 11th International Conference on Machine Learning and Applications.

[21]  Youngsu Park,et al.  Real-Time Defects Detection Algorithm for High-Speed Steel Bar in Coil , 2007 .

[22]  W. James MacLean,et al.  CCD noise removal in digital images , 2006, IEEE Transactions on Image Processing.

[23]  Haipeng Wang,et al.  Target Classification Using the Deep Convolutional Networks for SAR Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Yi Li,et al.  畳込みニューラルネットワークに基づくエンドツーエンド帯鋼表面欠陥認識システム【Powered by NICT】 , 2017 .

[25]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Chung-Feng Jeffrey Kuo,et al.  Integrating image processing and classification technology into automated polarizing film defect inspection , 2017 .

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Q. Peng,et al.  An improved Otsu method using the weighted object variance for defect detection , 2015 .

[29]  Franz Pernkopf,et al.  Image Acquisition Techniques for Automatic Visual Inspection of Metallic Surfaces , 2003 .

[30]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[31]  Ke Xu,et al.  Application of RNAMlet to surface defect identification of steels , 2018, Optics and Lasers in Engineering.

[32]  Antonia Moropoulou,et al.  Optical inspection for quantification of decay on stone surfaces , 2007 .

[33]  Dusmanta Kumar Mohanta,et al.  Review of vision-based steel surface inspection systems , 2014, EURASIP Journal on Image and Video Processing.

[34]  José R. Perán,et al.  Automated visual classification of frequent defects in flat steel coils , 2011 .

[35]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[36]  Kay Chen Tan,et al.  A Generic Deep-Learning-Based Approach for Automated Surface Inspection , 2018, IEEE Transactions on Cybernetics.

[37]  Jie Zhao,et al.  Steel surface defects recognition based on multi-type statistical features and enhanced twin support vector machine , 2017 .

[38]  J. López-Higuera,et al.  Real-time arc-welding defect detection and classification with principal component analysis and artificial neural networks , 2007 .