A Multilevel Information Fusion-Based Deep Learning Method for Vision-Based Defect Recognition

Vision-based defect recognition is an important technology to guarantee quality in modern manufacturing systems. Deep learning (DL) becomes a research hotspot in vision-based defect recognition due to outstanding performances. However, most of the DL methods require a large sample to learn the defect information. While in some real-world cases, it is difficult and costly for data collecting, and only a small sample is available. Generally, a small sample contains less information, which may mislead the DL models so that they cannot work as expected. Therefore, this requirement impedes the wide applications of DL in vision-based defect recognition. To overcome this problem, this article proposes a multilevel information fusion-based DL method for vision-based defect recognition. In the proposed method, a three-level Gaussian pyramid is introduced to generate multilevel information of the defect so that more information is available for model training. After the Gaussian pyramid, three VGG16 networks are built to learn the information and the outputs are fused for the final recognition result. The experimental results show that the proposed method can extract more useful information and achieve better performances on small-sample tasks, compared with the conventional DL methods and defect recognition methods. Furthermore, the analysis results of the robustness and response time also indicate that the proposed method is robust for the noise input, and it is fast for defect recognition, which takes 13.74 ms to handle a defect image.

[1]  Fumio Harashima,et al.  Development of an Automatic Stencil Inspection System Using Modified Hough Transform and Fuzzy Logic , 2007, IEEE Transactions on Industrial Electronics.

[2]  Xinyu Li,et al.  A semi-supervised convolutional neural network-based method for steel surface defect recognition , 2020, Robotics Comput. Integr. Manuf..

[3]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[4]  Edward J. Delp,et al.  Segmentation of textured images using a multiresolution Gaussian autoregressive model , 1999, IEEE Trans. Image Process..

[5]  Wen Chen,et al.  A New Ensemble Approach based on Deep Convolutional Neural Networks for Steel Surface Defect classification , 2018 .

[6]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[7]  Yundong Li,et al.  Deformable Patterned Fabric Defect Detection With Fisher Criterion-Based Deep Learning , 2017, IEEE Transactions on Automation Science and Engineering.

[8]  Yunhui Yan,et al.  A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects , 2013 .

[9]  Hong Zhang,et al.  Smooth Nonnegative Matrix Factorization for Defect Detection Using Microwave Nondestructive Testing and Evaluation , 2014, IEEE Transactions on Instrumentation and Measurement.

[10]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Hua Yang,et al.  An Unsupervised-Learning-Based Approach for Automated Defect Inspection on Textured Surfaces , 2018, IEEE Transactions on Instrumentation and Measurement.

[12]  Yong Huang,et al.  Texture decomposition by harmonics extraction from higher order statistics , 2004, IEEE Trans. Image Process..

[13]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[15]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[16]  Kay Chen Tan,et al.  A Generic Deep-Learning-Based Approach for Automated Surface Inspection , 2018, IEEE Transactions on Cybernetics.

[17]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[18]  Marco Vannucci,et al.  A fuzzy inference system applied to defect detection in flat steel production , 2010, International Conference on Fuzzy Systems.

[19]  Anil K. Jain,et al.  A Structural Approach To Identify Defects In Textured Images , 1988, Proceedings of the 1988 IEEE International Conference on Systems, Man, and Cybernetics.

[20]  Xianghua Xie,et al.  A Review of Recent Advances in Surface Defect Detection using Texture analysis Techniques , 2008 .

[21]  Zhigang Liu,et al.  Automatic Defect Detection of Fasteners on the Catenary Support Device Using Deep Convolutional Neural Network , 2018, IEEE Transactions on Instrumentation and Measurement.

[22]  Ling Li,et al.  Distributed defect recognition on steel surfaces using an improved random forest algorithm with optimal multi-feature-set fusion , 2018, Multimedia Tools and Applications.

[23]  W.K. Wong,et al.  A new intelligent fabric defect detection and classification system based on Gabor filter and modified Elman neural network , 2010, 2010 2nd International Conference on Advanced Computer Control.

[24]  Jacob Scharcanski,et al.  Stochastic texture analysis for monitoring stochastic processes in industry , 2005, Pattern Recognit. Lett..

[25]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[26]  Yixin Yin,et al.  Real-time Detection of Steel Strip Surface Defects Based on Improved YOLO Detection Network , 2018 .

[27]  Joachim M. Buhmann,et al.  Wheel Defect Detection With Machine Learning , 2018, IEEE Transactions on Intelligent Transportation Systems.

[28]  Ke Xu,et al.  Application of Shearlet transform to classification of surface defects for metals , 2015, Image Vis. Comput..

[29]  Shu Liao,et al.  Dominant Local Binary Patterns for Texture Classification , 2009, IEEE Transactions on Image Processing.

[30]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[31]  Jürgen Schmidhuber,et al.  Steel defect classification with Max-Pooling Convolutional Neural Networks , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[32]  K. F. C. Yiu,et al.  Fabric defect detection using morphological filters , 2009, Image Vis. Comput..

[33]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[34]  Edward H. Adelson,et al.  PYRAMID METHODS IN IMAGE PROCESSING. , 1984 .

[35]  Xinyu Li,et al.  A zero-shot learning method for fault diagnosis under unknown working loads , 2019, Journal of Intelligent Manufacturing.

[36]  T. Kurfess,et al.  Automatic thresholding for defect detection by background histogram mode extents , 2015 .

[37]  Karel J. Zuiderveld,et al.  Contrast Limited Adaptive Histogram Equalization , 1994, Graphics Gems.

[38]  Josep Tornero,et al.  On the detection of defects on specular car body surfaces , 2017 .

[39]  Yigang He,et al.  Generalized Completed Local Binary Patterns for Time-Efficient Steel Surface Defect Classification , 2019, IEEE Transactions on Instrumentation and Measurement.

[40]  M. Mirmehdi,et al.  TEXEMS: Texture Exemplars for Defect Detection on Random Textured Surfaces , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Li Yi,et al.  An evolutionary classifier for steel surface defects with small sample set , 2017, EURASIP J. Image Video Process..

[42]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).