Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture

Abstract In recent years, research has proposed several deep learning (DL) approaches to providing reliable remaining useful life (RUL) predictions in Prognostics and Health Management (PHM) applications. Although supervised DL techniques, such as Convolutional Neural Network and Long-Short Term Memory, have outperformed traditional prognosis algorithms, they are still dependent on large labeled training datasets. With respect to real-life PHM applications, high-quality labeled training data might be both challenging and time-consuming to acquire. Alternatively, unsupervised DL techniques introduce an initial pre-training stage to extract degradation related features from raw unlabeled training data automatically. Thus, the combination of unsupervised and supervised (semi-supervised) learning has the potential to provide high RUL prediction accuracy even with reduced amounts of labeled training data. This paper investigates the effect of unsupervised pre-training in RUL predictions utilizing a semi-supervised setup. Additionally, a Genetic Algorithm (GA) approach is applied in order to tune the diverse amount of hyper-parameters in the training procedure. The advantages of the proposed semi-supervised setup have been verified on the popular C-MAPSS dataset. The experimental study, compares this approach to purely supervised training, both when the training data is completely labeled and when the labeled training data is reduced, and to the most robust results in the literature. The results suggest that unsupervised pre-training is a promising feature in RUL predictions subjected to multiple operating conditions and fault modes.

[1]  David Haussler,et al.  Unsupervised learning of distributions on binary vectors using two layer networks , 1991, NIPS 1991.

[2]  Xue-wen Chen,et al.  Big Data Deep Learning: Challenges and Perspectives , 2014, IEEE Access.

[3]  Xiang Li,et al.  Remaining useful life estimation in prognostics using deep convolution neural networks , 2018, Reliab. Eng. Syst. Saf..

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[6]  Geoffrey E. Hinton,et al.  Training Recurrent Neural Networks , 2013 .

[7]  Li Lin,et al.  Remaining useful life estimation of engineered systems using vanilla LSTM neural networks , 2018, Neurocomputing.

[8]  Pascal Vincent,et al.  The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training , 2009, AISTATS.

[9]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[10]  Lin Ma,et al.  Prognostic modelling options for remaining useful life estimation by industry , 2011 .

[11]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[12]  Geoffrey E. Hinton A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[13]  Taehoon Lee,et al.  Semi-supervised Learning with Deep Generative Models for Asset Failure Prediction , 2017, ArXiv.

[14]  P.W. Kalgren,et al.  Defining PHM, A Lexical Evolution of Maintenance and Logistics , 2006, 2006 IEEE Autotestcon.

[15]  Abhinav Saxena,et al.  Damage propagation modeling for aircraft engine run-to-failure simulation , 2008, 2008 International Conference on Prognostics and Health Management.

[16]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[17]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[18]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[19]  Chetan Gupta,et al.  Long Short-Term Memory Network for Remaining Useful Life estimation , 2017, 2017 IEEE International Conference on Prognostics and Health Management (ICPHM).

[20]  Li Lin,et al.  Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network , 2016, 2016 IEEE International Conference on Aircraft Utility Systems (AUS).

[21]  Vincent Roberge,et al.  Comparison of Parallel Genetic Algorithm and Particle Swarm Optimization for Real-Time UAV Path Planning , 2013, IEEE Transactions on Industrial Informatics.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Guangquan Zhao,et al.  Research advances in fault diagnosis and prognostic based on deep learning , 2016, 2016 Prognostics and System Health Management Conference (PHM-Chengdu).

[25]  Josh Patterson,et al.  Deep Learning: A Practitioner's Approach , 2017 .

[26]  Charles C. Kemp,et al.  A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder , 2017, IEEE Robotics and Automation Letters.

[27]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[28]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Xiaoli Li,et al.  Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life , 2016, DASFAA.

[30]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[32]  Yanyang Zi,et al.  Switching State-Space Degradation Model With Recursive Filter/Smoother for Prognostics of Remaining Useful Life , 2019, IEEE Transactions on Industrial Informatics.

[33]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[34]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[35]  F.O. Heimes,et al.  Recurrent neural networks for remaining useful life estimation , 2008, 2008 International Conference on Prognostics and Health Management.