Self-supervised pretraining via contrast learning for intelligent incipient fault detection of bearings

Abstract Data-driven approaches for prognostic and health management (PHM) increasingly rely on massive historical data, yet annotations are expensive and time-consuming. Learning approaches that utilize semi-labeled or unlabeled data are becoming increasingly popular. In this paper, a self-supervised pre-training via contrast learning (SSPCL) is introduced to learn discriminative representations from unlabeled bearing datasets. Specifically, the SSPCL employs momentum contrast learning (MCL) to investigate the local representation in terms of instance-level discrimination contrast. Further, we propose a specific architecture for SSPCL deployment on bearing vibration signals by presenting several data augmentations for 1D sequences. On this basis, we put forward an incipient fault detection method based on SSPCL for run-to-failure cycle of rolling bearings. This approach transfers the SSPCL pre-trained model to a specific semi-supervised downstream task, effectively utilizing all unlabeled data and relying on only a little priori knowledge. A case study on FEMTO-ST datasets shows that the fine-tuned model is competent for incipient fault detection, outperforming other state-of-the-art methods. Furthermore, a supplemental case on a self-built fault datasets further demonstrate the great potential and superiority of our proposed SSPCL method in PHM.

[1]  Kaiming He,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Stella X. Yu,et al.  Unsupervised Feature Learning via Non-parametric Instance Discrimination , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Fang Duan,et al.  A Sparse Autoencoder-Based Unsupervised Scheme for Pump Fault Detection and Isolation , 2020 .

[4]  Jay Lee,et al.  Prognostics and health management design for rotary machinery systems—Reviews, methodology and applications , 2014 .

[5]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Maurizio Filippone,et al.  A comparative evaluation of outlier detection algorithms: Experiments and analyses , 2018, Pattern Recognit..

[7]  Yi Hu,et al.  Fault Detection and Identification Based on the Neighborhood Standardized Local Outlier Factor Method , 2013, Industrial & Engineering Chemistry Research.

[8]  Ruqiang Yan,et al.  Deep Learning Algorithms for Rotating Machinery Intelligent Diagnosis: An Open Source Benchmark Study , 2020, ISA transactions.

[9]  Xu Li,et al.  A multi-stage semi-supervised learning approach for intelligent fault diagnosis of rolling bearing using data augmentation and metric learning , 2021 .

[10]  Shaojiang Dong,et al.  Deep Residual Networks With Adaptively Parametric Rectifier Linear Units for Fault Diagnosis , 2021, IEEE Transactions on Industrial Electronics.

[11]  Yu Cheng,et al.  Early Fault Detection Approach With Deep Architectures , 2018, IEEE Transactions on Instrumentation and Measurement.

[12]  Xu Wang,et al.  Multi-scale deep intra-class transfer learning for bearing fault diagnosis , 2020, Reliab. Eng. Syst. Saf..

[13]  Jianyu Long,et al.  End-to-end unsupervised fault detection using a flow-based model , 2021, Reliab. Eng. Syst. Saf..

[14]  Alexei A. Efros,et al.  Colorful Image Colorization , 2016, ECCV.

[15]  Tommy W. S. Chow,et al.  Anomaly Detection and Fault Prognosis for Bearings , 2016, IEEE Transactions on Instrumentation and Measurement.

[16]  Michael Pecht,et al.  Deep Residual Shrinkage Networks for Fault Diagnosis , 2020, IEEE Transactions on Industrial Informatics.

[17]  Dana H. Ballard,et al.  Modular Learning in Neural Networks , 1987, AAAI.

[18]  Ali Razavi,et al.  Data-Efficient Image Recognition with Contrastive Predictive Coding , 2019, ICML.

[19]  Jongwon Seok,et al.  Bearing Fault Detection and Diagnosis Using Case Western Reserve University Dataset With Deep Learning Approaches: A Review , 2020, IEEE Access.

[20]  Zheng Liu,et al.  Semisupervised Deep Sparse Auto-Encoder With Local and Nonlocal Information for Intelligent Fault Diagnosis of Rotating Machinery , 2021, IEEE Transactions on Instrumentation and Measurement.

[21]  Yan Zhang,et al.  A hybrid classification autoencoder for semi-supervised fault diagnosis in rotating machinery , 2021 .

[22]  Weihua Li,et al.  Multisensor Feature Fusion for Bearing Fault Diagnosis Using Sparse Autoencoder and Deep Belief Network , 2017, IEEE Transactions on Instrumentation and Measurement.

[23]  Peng Ding,et al.  Statistical Alignment-Based Metagated Recurrent Unit for Cross-Domain Machinery Degradation Trend Prognostics Using Limited Data , 2021, IEEE Transactions on Instrumentation and Measurement.

[24]  Sergey Levine,et al.  Time-Contrastive Networks: Self-Supervised Learning from Video , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Jun Zhu,et al.  A new data-driven transferable remaining useful life prediction approach for bearing under different working conditions , 2020 .

[26]  Hichem Snoussi,et al.  Data-driven prognostic method based on self-supervised learning approaches for fault detection , 2018, J. Intell. Manuf..

[27]  Wei Song,et al.  A semi-supervised auto-encoder using label and sparse regularizations for classification , 2019, Appl. Soft Comput..

[28]  Fanming Meng,et al.  An Improved Bearing Fault Diagnosis Method using One-Dimensional CNN and LSTM , 2018, Strojniški vestnik - Journal of Mechanical Engineering.

[29]  Yongbo Li,et al.  Application of Bandwidth EMD and Adaptive Multiscale Morphology Analysis for Incipient Fault Diagnosis of Rolling Bearings , 2017, IEEE Transactions on Industrial Electronics.

[30]  Wentao Mao,et al.  A New Deep Dual Temporal Domain Adaptation Method for Online Detection of Bearings Early Fault , 2021, Entropy.

[31]  Xu Liu,et al.  A Predictive Fault Diagnose Method of Wind Turbine Based on K-means Clustering and Neural Networks , 2016 .

[32]  Bin Zhang,et al.  Bearing performance degradation assessment using long short-term memory recurrent network , 2019, Comput. Ind..

[33]  Jian Cheng,et al.  Symplectic weighted sparse support matrix machine for gear fault diagnosis , 2021 .

[34]  Shih-Fu Chang,et al.  Unsupervised Embedding Learning via Invariant and Spreading Instance Feature , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Wentao Mao,et al.  Online detection for bearing incipient fault based on deep transfer learning , 2020 .

[36]  Peng Ding,et al.  A Novel Remaining Useful Life Prediction Method of Rolling Bearings Based on Deep Transfer Auto-Encoder , 2021, IEEE Transactions on Instrumentation and Measurement.

[37]  Jie Zhou,et al.  Deep Transfer Metric Learning. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[38]  Bin Li,et al.  Early Fault Detection of Machine Tools Based on Deep Learning and Dynamic Identification , 2019, IEEE Transactions on Industrial Electronics.

[39]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[40]  Xianghong Cheng,et al.  A New Compressed-Structure-Based Coning Algorithm for Fiber Optic Strapdown Inertial Navigation Systems , 2021, IEEE Transactions on Instrumentation and Measurement.

[41]  Peng Ding,et al.  Mechatronics Equipment Performance Degradation Assessment Using Limited and Unlabeled Data , 2021, IEEE Transactions on Industrial Informatics.

[42]  Fillia Makedon,et al.  A Survey on Contrastive Self-supervised Learning , 2020, Technologies.

[43]  Jianzhong Zhou,et al.  Semi-supervised weighted kernel clustering based on gravitational search for fault diagnosis. , 2014, ISA transactions.

[44]  Gian Antonio Susto,et al.  An Explainable Artificial Intelligence Approach for Unsupervised Fault Detection and Diagnosis in Rotating Machinery , 2021, ArXiv.

[45]  Kaixiang Peng,et al.  A Health Indicator Construction Method based on Deep Belief Network for Remaining Useful Life Prediction , 2019, 2019 Prognostics and System Health Management Conference (PHM-Qingdao).

[46]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[47]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[48]  Aapo Hyvärinen,et al.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models , 2010, AISTATS.

[49]  Jiafu Wan,et al.  Modified Stacked Autoencoder Using Adaptive Morlet Wavelet for Intelligent Fault Diagnosis of Rotating Machinery , 2021, IEEE/ASME Transactions on Mechatronics.

[50]  Minping Jia,et al.  Remaining Useful Life Estimation Under Multiple Operating Conditions via Deep Subdomain Adaptation , 2021, IEEE Transactions on Instrumentation and Measurement.

[51]  Xiaodong Liang,et al.  An Effective Induction Motor Fault Diagnosis Approach Using Graph-Based Semi-Supervised Learning , 2021, IEEE Access.

[52]  Fan Xu,et al.  Life prediction of lithium-ion batteries based on stacked denoising autoencoders , 2021, Reliab. Eng. Syst. Saf..