Visual inspection of steel surface defects based on domain adaptation and adaptive convolutional neural network

Abstract Automatic inspection methods based on machine vision have been widely employed for steel surface defect detection. The central purpose of these methods is to extract features to represent different defects. However, current methods depend on machine learning that demands handcrafted features and overlooks the domain shift. In this paper, we propose a new method combining domain adaptation (DA) and adaptive convolutional neural network (ACNN), called DA-ACNN, to achieve steel surface defect detection. The convolutional neural network (CNN) is used as the backbone. To account for the lack of labels in a new domain, we introduce an additional domain classifier and a constraint on label probability distribution to achieve the cross-domain and cross-task recognition. The normal distribution and the quadratic function are used to optimize the loss to improve the network performance. Adaptive learning rates based on the loss and the weight, respectively, are proposed to minimize the losses of DA and classification. We conducted experiments on steel surface defect datasets to validate the effectiveness of DA-ACNN. Compared with the classical CNN and other approaches, the results demonstrate the superiority of the proposed method.

[1]  Shunming Li,et al.  A New Transfer Learning Method and its Application on Rotating Machine Fault Diagnosis Under Variant Working Conditions , 2018, IEEE Access.

[2]  Zhong Chen,et al.  Realtime in-plane displacements tracking of the precision positioning stage based on computer micro-vision , 2019 .

[3]  Anirban Mukherjee,et al.  Automatic Defect Detection on Hot-Rolled Flat Steel Products , 2013, IEEE Transactions on Instrumentation and Measurement.

[4]  Jingfeng Jiang,et al.  Deep learning for the identification of bruised apples by fusing 3D deep features for apple grading systems , 2020 .

[5]  Keith Worden,et al.  On the application of domain adaptation in structural health monitoring , 2020, Mechanical Systems and Signal Processing.

[6]  Wenjing Ye,et al.  Damage localization in plate-like structures using time-varying feature and one-dimensional convolutional neural network , 2021 .

[7]  Mei Wang,et al.  Deep Visual Domain Adaptation: A Survey , 2018, Neurocomputing.

[8]  Yanning Zhang,et al.  An unsupervised deep domain adaptation approach for robust speech recognition , 2017, Neurocomputing.

[9]  Wei Zhang,et al.  Multi-Layer domain adaptation method for rolling bearing fault diagnosis , 2019, Signal Process..

[10]  Xin He,et al.  LLR: Learning learning rates by LSTM for training neural networks , 2020, Neurocomputing.

[11]  Oscar Castillo,et al.  Comparison of particle swarm optimization variants with fuzzy dynamic parameter adaptation for modular granular neural networks for human recognition , 2020, J. Intell. Fuzzy Syst..

[12]  Yu Zhou,et al.  Application of neural network algorithm in fault diagnosis of mechanical intelligence , 2020, Mechanical Systems and Signal Processing.

[13]  Yu He,et al.  PGA-Net: Pyramid Feature Fusion and Global Context Attention Network for Automated Surface Defect Detection , 2020, IEEE Transactions on Industrial Informatics.

[14]  Oscar Castillo,et al.  A Grey Wolf Optimizer for Modular Granular Neural Networks for Human Recognition , 2017, Comput. Intell. Neurosci..

[15]  Yu He,et al.  Semi-supervised defect classification of steel surface based on multi-training and generative adversarial network , 2019, Optics and Lasers in Engineering.

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Jinrui Wang,et al.  A novel geodesic flow kernel based domain adaptation approach for intelligent fault diagnosis under varying working condition , 2020, Neurocomputing.

[18]  Yigang He,et al.  A cost-effective and automatic surface defect inspection system for hot-rolled flat steel , 2016 .

[19]  Zhihai Lu,et al.  Pathological brain detection based on AlexNet and transfer learning , 2019, J. Comput. Sci..

[20]  Hui Li,et al.  An active learning method combining deep neural network and weighted sampling for structural reliability analysis , 2020 .

[21]  Qing Li,et al.  Judgment and optimization of video image recognition in obstacle detection in intelligent vehicle , 2020 .

[22]  Di He,et al.  Defect detection of hot rolled steels with a new object detection framework called classification priority network , 2019, Comput. Ind. Eng..

[23]  Liming Liu,et al.  Multi-class classification method using twin support vector machines with multi-information for steel surface defects , 2018 .

[24]  Hongkun Wu,et al.  Damage detection techniques for wind turbine blades: A review , 2020 .

[25]  Yaguo Lei,et al.  Applications of machine learning to machine fault diagnosis: A review and roadmap , 2020 .

[26]  M. Matuszewski,et al.  Using artificial intelligence models for the prediction of surface wear based on surface isotropy levels , 2018, Robotics and Computer-Integrated Manufacturing.

[27]  Haidong Shao,et al.  Rolling bearing fault feature learning using improved convolutional deep belief network with compressed sensing , 2018 .

[28]  Guohui Tian,et al.  A novel scene classification model combining ResNet based transfer learning and data augmentation with a filter , 2019, Neurocomputing.

[29]  Teik C. Lim,et al.  Pure electric vehicle nonstationary interior sound quality prediction based on deep CNNs with an adaptable learning rate tree , 2021 .

[30]  A. O. Martins Luiz,et al.  Automatic detection of surface defects on rolled steel using Computer Vision and Artificial Neural Networks , 2010, IECON 2010 - 36th Annual Conference on IEEE Industrial Electronics Society.

[31]  Liqiang Nie,et al.  Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm , 2016, Neurocomputing.

[32]  Claudia I. González,et al.  Optimal Recognition Model Based on Convolutional Neural Networks and Fuzzy Gravitational Search Algorithm Method , 2020, Hybrid Intelligent Systems in Control, Pattern Recognition and Medicine.

[33]  Yunhui Yan,et al.  Steel Surface Defect Classification Based on Discriminant Manifold Regularized Local Descriptor , 2018, IEEE Access.

[34]  Haibo He,et al.  A Hierarchical Deep Domain Adaptation Approach for Fault Diagnosis of Power Plant Thermal System , 2019, IEEE Transactions on Industrial Informatics.

[35]  Claudia I. González,et al.  Filter Size Optimization on a Convolutional Neural Network Using FGSA , 2020, Intuitionistic and Type-2 Fuzzy Logic Enhancements in Neural and Optimization Algorithms.

[36]  Youping Chen,et al.  Classification of surface defects on steel sheet using convolutional neural networks , 2017 .

[37]  Bin Yang,et al.  An intelligent fault diagnosis approach based on transfer learning from laboratory bearings to locomotive bearings , 2019, Mechanical Systems and Signal Processing.

[38]  Michael Ying Yang,et al.  A deep-learning-based approach for fast and robust steel surface defects classification , 2019, Optics and Lasers in Engineering.

[39]  Yao Feng,et al.  A multi-class classifier based on support vector hyper-spheres for steel plate surface defects , 2019, Chemometrics and Intelligent Laboratory Systems.