A New Parameter Repurposing Method for Parameter Transfer With Small Dataset and Its Application in Fault Diagnosis of Rolling Element Bearings

Transfer learning is a promising deep learning approach that can be used in applications that suffer from insufficient training data. Parameter transfer, which is a method of improving the accuracy and training speed by training the target network using the parameters of the source network, selects two conventional parameter repurposing methods, such as parameter freezing and fine-tuning depending on the amount of target data and network size. In this paper, we propose a novel method to increase the performance in the target domain in an intermediate approach of both methods. The proposed method, selective parameter freezing (SPF), freezes only a portion of parameters not freeze or fine-tunes all parameters within a layer by choosing output-sensitive parameters from the source network. Freezing only sensitive parameters while training is to reduce the amount of trainable parameter and protect informative parameters from overfitting to a small number of target data. Using two sets of the source–target domain, artificial faults with different fault size and artificial faults—natural faults of rolling element bearing, the proposed SPF allows adaptation to the target domain by choosing the best degree of freezing with various amounts of target data and size of networks compared to conventional approaches.

[1]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Wei Zhang,et al.  Multi-Layer domain adaptation method for rolling bearing fault diagnosis , 2019, Signal Process..

[3]  Shunming Li,et al.  A New Transfer Learning Method and its Application on Rotating Machine Fault Diagnosis Under Variant Working Conditions , 2018, IEEE Access.

[4]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[5]  Bo Geng,et al.  DAML: Domain Adaptation Metric Learning , 2011, IEEE Transactions on Image Processing.

[6]  Uwe Stilla,et al.  Deep Learning Earth Observation Classification Using ImageNet Pretrained Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[7]  Jude Shavlik,et al.  Chapter 11 Transfer Learning , 2009 .

[8]  Lin Lin,et al.  A novel gas turbine fault diagnosis method based on transfer learning with CNN , 2019, Measurement.

[9]  Chang Wang,et al.  Heterogeneous Domain Adaptation Using Manifold Alignment , 2011, IJCAI.

[10]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[11]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[12]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[13]  Wei Zhang,et al.  A robust intelligent fault diagnosis method for rolling element bearings based on deep distance metric learning , 2018, Neurocomputing.

[14]  Wei Li,et al.  Bearing Fault Diagnosis Based on Domain Adaptation Using Transferable Features under Different Working Conditions , 2018, Shock and Vibration.

[15]  Hyunseok Oh,et al.  Scalable and Unsupervised Feature Engineering Using Vibration-Imaging and Deep Learning for Rotor System Diagnosis , 2018, IEEE Transactions on Industrial Electronics.

[16]  Yaguo Lei,et al.  Deep Convolutional Transfer Learning Network: A New Method for Intelligent Fault Diagnosis of Machines With Unlabeled Data , 2019, IEEE Transactions on Industrial Electronics.

[17]  Bin Yang,et al.  An intelligent fault diagnosis approach based on transfer learning from laboratory bearings to locomotive bearings , 2019, Mechanical Systems and Signal Processing.

[18]  Ran Zhang,et al.  Transfer Learning With Neural Networks for Bearing Fault Diagnosis in Changing Working Conditions , 2017, IEEE Access.

[19]  Noureddine Zerhouni,et al.  Strategies to Face Imbalanced and Unlabelled Data in Phm Applications , 2013 .

[20]  Tao Zhang,et al.  Deep Model Based Domain Adaptation for Fault Diagnosis , 2017, IEEE Transactions on Industrial Electronics.

[21]  Ruqiang Yan,et al.  Highly Accurate Machine Fault Diagnosis Using Deep Transfer Learning , 2019, IEEE Transactions on Industrial Informatics.

[22]  Stefan Winkler,et al.  Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning , 2015, ICMI.

[23]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[24]  Frank C. Park,et al.  Transfer learning for automated optical inspection , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[25]  Gaoliang Peng,et al.  A deep convolutional neural network with new training methods for bearing fault diagnosis under noisy environment and different working load , 2018, Mechanical Systems and Signal Processing.

[26]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[27]  Xiang Li,et al.  Cross-Domain Fault Diagnosis of Rolling Element Bearings Using Deep Generative Neural Networks , 2019, IEEE Transactions on Industrial Electronics.

[28]  Jiong Tang,et al.  Preprocessing-Free Gear Fault Diagnosis Using Small Datasets With Deep Convolutional Neural Network-Based Transfer Learning , 2017, IEEE Access.

[29]  Wei Zhang,et al.  A New Deep Learning Model for Fault Diagnosis with Good Anti-Noise and Domain Adaptation Ability on Raw Vibration Signals , 2017, Sensors.

[30]  Bo Zhang,et al.  Intelligent Fault Diagnosis Under Varying Working Conditions Based on Domain Adaptive Convolutional Neural Networks , 2018, IEEE Access.

[31]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[33]  Robert X. Gao,et al.  Deep learning and its applications to machine health monitoring , 2019, Mechanical Systems and Signal Processing.

[34]  Bo Zhang,et al.  Bearing Fault Diagnosis Under Variable Working Conditions Based on Domain Adaptation Using Feature Transfer Learning , 2018, IEEE Access.

[35]  Trevor Darrell,et al.  What you saw is not what you get: Domain adaptation using asymmetric kernel transforms , 2011, CVPR 2011.

[36]  Qiang Yang,et al.  Heterogeneous Transfer Learning for Image Classification , 2011, AAAI.

[37]  Ivor W. Tsang,et al.  Learning With Augmented Features for Supervised and Semi-Supervised Heterogeneous Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.