A New Snapshot Ensemble Convolutional Neural Network for Fault Diagnosis

Deep learning (DL) has been widely applied in fault diagnosis field. The ensemble of DL can significantly improve the generalization ability of DL. Snapshot ensemble learning, which uses the cyclic learning rate scheduler (CLR) and then combines the local minima in once training to form an ensemble method, has shown powerful performance. However, the learning rate range of CLR should be pre-defined by experiences, which may limit its prediction accuracy. In this paper, a new snapshot ensemble convolutional neural network (SECNN) is proposed, which can find the proper range of learning rate for SECNN automatically when facing a new dataset. First, a max–min cosine cyclic learning rate scheduler (MMCCLR) is designed to avoid learning rate range being affected by other parameters. Then, a new learning rate testing (logLR Test) is applied to estimate the proper learning rate range for MMCCLR. Finally, the SECNN with MMCCLR is developed. The SECNN is tested on three famous datasets, including bearing dataset of Case Western Reserve University, self-priming centrifugal pump dataset, and bearing dataset provided by the Machinery Failure Prevention Technology. The results have been improved a lot by the proposed SECNN methods. The SECNN has also achieved the start-of-the-art prediction accuracies by comparing with other DL and traditional methods.

[1]  Haidong Shao,et al.  Rolling bearing fault diagnosis using an optimization deep belief network , 2015 .

[2]  Changqing Shen,et al.  Stacked Sparse Autoencoder-Based Deep Network for Fault Diagnosis of Rotating Machinery , 2017, IEEE Access.

[3]  Jianbin Qiu,et al.  Fault Detection for Nonlinear Process With Deterministic Disturbances: A Just-In-Time Learning Based Data Driven Method , 2017, IEEE Transactions on Cybernetics.

[4]  Zhenyu Wu,et al.  An Integrated Ensemble Learning Model for Imbalanced Fault Diagnostics and Prognostics , 2018, IEEE Access.

[5]  Chao Li,et al.  Shared Predictive Cross-Modal Deep Quantization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Shunming Li,et al.  A New Transfer Learning Method and its Application on Rotating Machine Fault Diagnosis Under Variant Working Conditions , 2018, IEEE Access.

[7]  Yang Xiao,et al.  Fault Diagnosis Using a Joint Model Based on Sparse Representation and SVM , 2016, IEEE Transactions on Instrumentation and Measurement.

[8]  Yi Qin,et al.  The Optimized Deep Belief Networks With Improved Logistic Sigmoid Units and Their Application in Fault Diagnosis for Planetary Gearboxes of Wind Turbines , 2019, IEEE Transactions on Industrial Electronics.

[9]  Zhibin Zhao,et al.  Sparse Deep Stacking Network for Fault Diagnosis of Motor , 2018, IEEE Transactions on Industrial Informatics.

[10]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Bo Zhang,et al.  Intelligent Fault Diagnosis Under Varying Working Conditions Based on Domain Adaptive Convolutional Neural Networks , 2018, IEEE Access.

[12]  Frank Hutter,et al.  SGDR: Stochastic Gradient Descent with Warm Restarts , 2016, ICLR.

[13]  Jiangtao Wen,et al.  Intelligent Bearing Fault Diagnosis Method Combining Compressed Data Acquisition and Deep Learning , 2018, IEEE Transactions on Instrumentation and Measurement.

[14]  Liang Gao,et al.  A New Deep Transfer Learning Based on Sparse Auto-Encoder for Fault Diagnosis , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[15]  Lalu Mansinha,et al.  Localization of the complex spectrum: the S transform , 1996, IEEE Trans. Signal Process..

[16]  Leslie N. Smith,et al.  Cyclical Learning Rates for Training Neural Networks , 2015, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[17]  Miao He,et al.  Deep Learning Based Approach for Bearing Fault Diagnosis , 2017, IEEE Transactions on Industry Applications.

[18]  Liang Gao,et al.  A new subset based deep feature learning method for intelligent fault diagnosis of bearing , 2018, Expert Syst. Appl..

[19]  Zhiwei Gao,et al.  From Model, Signal to Knowledge: A Data-Driven Perspective of Fault Detection and Diagnosis , 2013, IEEE Transactions on Industrial Informatics.

[20]  Chen Lu,et al.  Fault Diagnosis for Rotating Machinery: A Method based on Image Processing , 2016, PloS one.

[21]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Enrico Zio,et al.  Artificial intelligence for fault diagnosis of rotating machinery: A review , 2018, Mechanical Systems and Signal Processing.

[23]  Min Xia,et al.  Fault Diagnosis for Rotating Machinery Using Multiple Sensors and Convolutional Neural Networks , 2018, IEEE/ASME Transactions on Mechatronics.

[24]  Wei-Yun Yau,et al.  Structured AutoEncoders for Subspace Clustering , 2018, IEEE Transactions on Image Processing.

[25]  Mohammad Modarres,et al.  Deep Learning Enabled Fault Diagnosis Using Time-Frequency Image Analysis of Rolling Element Bearings , 2017 .

[26]  Weihua Li,et al.  Multisensor Feature Fusion for Bearing Fault Diagnosis Using Sparse Autoencoder and Deep Belief Network , 2017, IEEE Transactions on Instrumentation and Measurement.

[27]  Gaoliang Peng,et al.  A deep convolutional neural network with new training methods for bearing fault diagnosis under noisy environment and different working load , 2018, Mechanical Systems and Signal Processing.

[28]  Robert X. Gao,et al.  Deep learning and its applications to machine health monitoring , 2019, Mechanical Systems and Signal Processing.

[29]  Tao Zhang,et al.  Fault Diagnosis for Rotating Machinery Based on Convolutional Neural Network and Empirical Mode Decomposition , 2017 .

[30]  Kilian Q. Weinberger,et al.  Snapshot Ensembles: Train 1, get M for free , 2017, ICLR.

[31]  Cong Wang,et al.  Construction of hierarchical diagnosis network based on deep learning and its application in the fault pattern recognition of rolling element bearings , 2016 .

[32]  Wei Liu,et al.  Pairwise Relationship Guided Deep Hashing for Cross-Modal Retrieval , 2017, AAAI.

[33]  Jimi Tjong,et al.  Automotive Internal-Combustion-Engine Fault Detection and Classification Using Artificial Neural Network Techniques , 2015, IEEE Transactions on Vehicular Technology.

[34]  Liang Chen,et al.  Hierarchical adaptive deep convolution neural network and its application to bearing fault diagnosis , 2016 .

[35]  Peng Wang,et al.  An Adaptive Multi-Sensor Data Fusion Method Based on Deep Convolutional Neural Networks for Fault Diagnosis of Planetary Gearbox , 2017, Sensors.

[36]  Zhenyu Huang,et al.  Multiple Marginal Fisher Analysis , 2019, IEEE Transactions on Industrial Electronics.

[37]  Haidong Shao,et al.  A novel method for intelligent fault diagnosis of rolling bearings using ensemble deep auto-encoders , 2018 .

[38]  Lin Zhao,et al.  A Deep Feature Optimization Fusion Method for Extracting Bearing Degradation Features , 2018, IEEE Access.

[39]  Wei Zhang,et al.  A New Deep Learning Model for Fault Diagnosis with Good Anti-Noise and Domain Adaptation Ability on Raw Vibration Signals , 2017, Sensors.

[40]  Xinbo Gao,et al.  Triplet-Based Deep Hashing Network for Cross-Modal Retrieval , 2018, IEEE Transactions on Image Processing.

[41]  Haidong Shao,et al.  Rolling bearing fault feature learning using improved convolutional deep belief network with compressed sensing , 2018 .

[42]  Giansalvo Cirrincione,et al.  Bearing Fault Detection by a Novel Condition-Monitoring Scheme Based on Statistical-Time Features and Neural Networks , 2013, IEEE Transactions on Industrial Electronics.

[43]  Steven X. Ding,et al.  A Review on Basic Data-Driven Approaches for Industrial Process Monitoring , 2014, IEEE Transactions on Industrial Electronics.

[44]  Dazhong Wu,et al.  Deep learning for smart manufacturing: Methods and applications , 2018, Journal of Manufacturing Systems.

[45]  Huanhuan Chen,et al.  Regularized Negative Correlation Learning for Neural Network Ensembles , 2009, IEEE Transactions on Neural Networks.