Remaining Useful Lifetime Prediction via Deep Domain Adaptation

In Prognostics and Health Management (PHM) sufficient prior observed degradation data is usually critical for Remaining Useful Lifetime (RUL) prediction. Most previous data-driven prediction methods assume that training (source) and testing (target) condition monitoring data have similar distributions. However, due to different operating conditions, fault modes, noise and equipment updates distribution shift exists across different data domains. This shift reduces the performance of predictive models previously built to specific conditions when no observed run-to-failure data is available for retraining. To address this issue, this paper proposes a new data-driven approach for domain adaptation in prognostics using Long Short-Term Neural Networks (LSTM). We use a time window approach to extract temporal information from time-series data in a source domain with observed RUL values and a target domain containing only sensor information. We propose a Domain Adversarial Neural Network (DANN) approach to learn domain-invariant features that can be used to predict the RUL in the target domain. The experimental results show that the proposed method can provide more reliable RUL predictions under datasets with different operating conditions and fault modes. These results suggest that the proposed method offers a promising approach to performing domain adaptation in practical PHM applications.

[1]  Zhigang Tian,et al.  An artificial neural network method for remaining useful life prediction of equipment subject to condition monitoring , 2012, J. Intell. Manuf..

[2]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[3]  Manuel Esperon-Miguez,et al.  A review of physics-based models in prognostics: Application to gears and bearings of rotating machinery , 2016 .

[4]  François Laviolette,et al.  Domain-Adversarial Neural Networks , 2014, ArXiv.

[5]  Shaojiang Dong,et al.  Bearing degradation process prediction based on the PCA and optimized LS-SVM model , 2013 .

[6]  Xiang Li,et al.  Remaining useful life estimation in prognostics using deep convolution neural networks , 2018, Reliab. Eng. Syst. Saf..

[7]  Kai Goebel,et al.  Damage Propagation Modeling for Aircraft Engine Prognostics , 2008 .

[8]  E. Bechhoefer,et al.  Development and Validation of Bearing Diagnostic and Prognostic Tools using HUMS Condition Indicators , 2008, 2008 IEEE Aerospace Conference.

[9]  Daming Lin,et al.  A review on machinery diagnostics and prognostics implementing condition-based maintenance , 2006 .

[10]  Mehryar Mohri,et al.  Domain adaptation and sample bias correction theory and algorithm for regression , 2014, Theor. Comput. Sci..

[11]  Lixiang Duan,et al.  On cross-domain feature fusion in gearbox fault diagnosis under various operating conditions based on Transfer Component Analysis , 2016, 2016 IEEE International Conference on Prognostics and Health Management (ICPHM).

[12]  Noureddine Zerhouni,et al.  Prognostics and Health Management for Maintenance Practitioners - Review, Implementation and Tools Evaluation , 2020, International Journal of Prognostics and Health Management.

[13]  George Chryssolouris,et al.  An approach to operational aircraft maintenance planning , 2010, Decis. Support Syst..

[14]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[15]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[16]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[17]  Li Lin,et al.  Remaining useful life estimation of engineered systems using vanilla LSTM neural networks , 2018, Neurocomputing.

[18]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[19]  S. Purushotham,et al.  Variational Adversarial Deep Domain Adaptation for Health Care Time Series Analysis , 2016 .

[20]  ChengXiang Zhai,et al.  Instance Weighting for Domain Adaptation in NLP , 2007, ACL.

[21]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[22]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Yuxin Cui,et al.  Transfer Learning with Deep Recurrent Neural Networks for Remaining Useful Life Estimation , 2018, Applied Sciences.

[24]  P J Webros BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[25]  Philip S. Yu,et al.  Transfer Feature Learning with Joint Distribution Adaptation , 2013, 2013 IEEE International Conference on Computer Vision.

[26]  Noureddine Zerhouni,et al.  Remaining useful life estimation based on nonlinear feature reduction and support vector regression , 2013, Eng. Appl. Artif. Intell..

[27]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[28]  Wei Zhang,et al.  A New Deep Learning Model for Fault Diagnosis with Good Anti-Noise and Domain Adaptation Ability on Raw Vibration Signals , 2017, Sensors.

[29]  Kate Saenko,et al.  Deep CORAL: Correlation Alignment for Deep Domain Adaptation , 2016, ECCV Workshops.

[30]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[31]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[33]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[34]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[35]  Oliver Schütze,et al.  A Neural Network-Evolutionary Computational Framework for Remaining Useful Life Estimation of Mechanical Systems , 2019, Neural Networks.

[36]  Barbara Hammer,et al.  Unsupervised Transfer Learning for Time Series via Self-Predictive Modelling - First Results , 2017 .

[37]  Xiang Li,et al.  Cross-Domain Fault Diagnosis of Rolling Element Bearings Using Deep Generative Neural Networks , 2019, IEEE Transactions on Industrial Electronics.

[38]  Bart De Schutter,et al.  Deep convolutional neural networks for detection of rail surface defects , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[39]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[40]  Chetan Gupta,et al.  Long Short-Term Memory Network for Remaining Useful Life estimation , 2017, 2017 IEEE International Conference on Prognostics and Health Management (ICPHM).

[41]  Li Lin,et al.  Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network , 2016, 2016 IEEE International Conference on Aircraft Utility Systems (AUS).

[42]  Lifeng Xi,et al.  Residual life predictions for ball bearings based on self-organizing map and back propagation neural network methods , 2007 .

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[44]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[45]  Houxiang Zhang,et al.  Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture , 2019, Reliab. Eng. Syst. Saf..

[46]  Edwin Lughofer,et al.  Domain-Invariant Partial-Least-Squares Regression. , 2018, Analytical chemistry.

[47]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[48]  Donghua Zhou,et al.  Remaining useful life estimation - A review on the statistical data driven approaches , 2011, Eur. J. Oper. Res..

[49]  Tao Zhang,et al.  Deep Model Based Domain Adaptation for Fault Diagnosis , 2017, IEEE Transactions on Industrial Electronics.

[50]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[51]  Bernhard Schölkopf,et al.  Semi-Supervised Domain Adaptation with Non-Parametric Copulas , 2012, NIPS.

[52]  Xiaoli Li,et al.  Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life , 2016, DASFAA.

[53]  Yili Hong,et al.  Prediction of remaining life of power transformers based on left truncated and right censored lifetime data , 2009, 0908.2901.

[54]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[55]  Yaguo Lei,et al.  Machinery health prognostics: A systematic review from data acquisition to RUL prediction , 2018 .

[56]  Enrico Zio,et al.  A data-driven fuzzy approach for predicting the remaining useful life in dynamic failure scenarios of a nuclear system , 2010, Reliab. Eng. Syst. Saf..

[57]  Md. Zakir Hossain,et al.  A Comprehensive Survey of Deep Learning for Image Captioning , 2018, ACM Comput. Surv..

[58]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[59]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[60]  F.O. Heimes,et al.  Recurrent neural networks for remaining useful life estimation , 2008, 2008 International Conference on Prognostics and Health Management.

[61]  Trevor Darrell,et al.  Deep Domain Confusion: Maximizing for Domain Invariance , 2014, CVPR 2014.

[62]  Wei Zhang,et al.  Multi-Layer domain adaptation method for rolling bearing fault diagnosis , 2019, Signal Process..