Transfer learning for Remaining Useful Life Prediction Based on Consensus Self-Organizing Models

The traditional paradigm for developing machine prognostics usually relies on generalization from data acquired in experiments under controlled conditions prior to deployment of the equipment. Detecting or predicting failures and estimating machine health in this way assumes that future field data will have a very similar distribution to the experiment data. However, many complex machines operate under dynamic environmental conditions and are used in many different ways. This makes collecting comprehensive data very challenging, and the assumption that pre-deployment data and post-deployment data follow very similar distributions is unlikely to hold. Transfer Learning (TL) refers to methods for transferring knowledge learned in one setting (the source domain) to another setting (the target domain). In this work, we present a TL method for predicting Remaining Useful Life (RUL) of equipment, under the assumption that labels are available only for the source domain and not the target domain. This setting corresponds to generalizing from a limited number of run-to-failure experiments performed prior to deployment into making prognostics with data coming from deployed equipment that is being used under multiple new operating conditions and experiencing previously unseen faults. We employ a deviation detection method, Consensus Self-Organizing Models (COSMO), to create transferable features for building the RUL regression model. These features capture how different target equipment is in comparison to its peers. The efficiency of the proposed TL method is demonstrated using the NASA Turbofan Engine Degradation Simulation Data Set. Models using the COSMO transferable features show better performance than other methods on predicting RUL when the target domain is more complex than the source domain.

[1]  Alaa Elwany,et al.  Residual Life Predictions in the Absence of Prior Degradation Knowledge , 2009, IEEE Transactions on Reliability.

[2]  Jay Lee,et al.  Prognostics and health management design for rotary machinery systems—Reviews, methodology and applications , 2014 .

[3]  Xiaodong Liu,et al.  A data-driven prognostics approach for RUL based on principle component and instance learning , 2016, ICPHM.

[4]  Enrico Zio,et al.  Echo State Network for the Remaining Useful Life Prediction of a Turbofan Engine , 2016, PHM Society European Conference.

[5]  Qiang Yang,et al.  Transfer Learning via Dimensionality Reduction , 2008, AAAI.

[6]  F.O. Heimes,et al.  Recurrent neural networks for remaining useful life estimation , 2008, 2008 International Conference on Prognostics and Health Management.

[7]  Abhinav Saxena,et al.  Damage propagation modeling for aircraft engine run-to-failure simulation , 2008, 2008 International Conference on Prognostics and Health Management.

[8]  Gregoris Mentzas,et al.  Review, analysis and synthesis of prognostic-based decision support methods for condition based maintenance , 2018, J. Intell. Manuf..

[9]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[10]  Nan Chen,et al.  Prognostics and Health Management: A Review on Data Driven Approaches , 2015 .

[11]  Shunming Li,et al.  A novel transfer learning method for robust fault diagnosis of rotating machines under variable working conditions , 2019, Measurement.

[12]  Uzay Kaymak,et al.  Remaining Useful Lifetime Prediction via Deep Domain Adaptation , 2019, Reliab. Eng. Syst. Saf..

[13]  Chao Liu,et al.  Deep Transfer Network with Joint Distribution Adaptation: A New Intelligent Fault Diagnosis Framework for Industry Application , 2018, ISA transactions.

[14]  Houxiang Zhang,et al.  Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture , 2019, Reliab. Eng. Syst. Saf..

[15]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[16]  Nathan L. Clarke,et al.  Fast Predictive Maintenance in Industrial Internet of Things (IIoT) with Deep Learning (DL): A Review , 2019, CERC.

[17]  Slawomir Nowaczyk,et al.  Self-monitoring for maintenance of vehicle fleets , 2017, Data Mining and Knowledge Discovery.

[18]  Brigitte Chebel-Morello,et al.  RUL prediction based on a new similarity-instance based approach , 2014, 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE).

[19]  Olga Fink,et al.  Fleet PHM for Critical Systems: Bi-level Deep Learning Approach for Fault Detection , 2018 .

[20]  Bin Yang,et al.  A Transfer Learning Method for Intelligent Fault Diagnosis from Laboratory Machines to Real-Case Machines , 2018, 2018 International Conference on Sensing,Diagnostics, Prognostics, and Control (SDPC).

[21]  Raymond J. Mooney,et al.  Transfer Learning by Mapping with Minimal Target Data , 2008 .

[22]  Stefan Byttner,et al.  Consensus self-organized models for fault detection (COSMO) , 2011, Eng. Appl. Artif. Intell..

[23]  M. S. Lebold,et al.  Hybrid reasoning for prognostic learning in CBM systems , 2001, 2001 IEEE Aerospace Conference Proceedings (Cat. No.01TH8542).

[24]  M. Pecht,et al.  Failure mechanisms based prognostics , 2008, 2008 International Conference on Prognostics and Health Management.

[25]  Huihui Miao,et al.  Joint Learning of Degradation Assessment and RUL Prediction for Aeroengines via Dual-Task Deep LSTM Networks , 2019, IEEE Transactions on Industrial Informatics.

[26]  Bin Liang,et al.  Remaining useful life prediction of aircraft engine based on degradation pattern learning , 2017, Reliab. Eng. Syst. Saf..

[27]  Jing Zhang,et al.  Joint Geometrical and Statistical Alignment for Visual Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Ying Chen,et al.  Performance-Based Gas Turbine Health Monitoring, Diagnostics, and Prognostics: A Survey , 2018, IEEE Transactions on Reliability.

[29]  Olga Fink,et al.  Domain Adaptive Transfer Learning for Fault Diagnosis , 2019, 2019 Prognostics and System Health Management Conference (PHM-Paris).

[30]  Stefan Byttner,et al.  Estimating p-Values for Deviation Detection , 2014, 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems.

[31]  Brian A. Weiss,et al.  A review of diagnostic and prognostic capabilities and best practices for manufacturing , 2019, J. Intell. Manuf..

[32]  Xiang Li,et al.  Remaining useful life estimation in prognostics using deep convolution neural networks , 2018, Reliab. Eng. Syst. Saf..

[33]  Fei Shen,et al.  Bearing fault diagnosis based on SVD feature extraction and transfer learning classification , 2015, 2015 Prognostics and System Health Management Conference (PHM).

[34]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.

[35]  Takehisa Yairi,et al.  A review on the application of deep learning in system health management , 2018, Mechanical Systems and Signal Processing.

[36]  Thorsteinn Rögnvaldsson,et al.  Predicting Air Compressor Failures with Echo State Networks , 2016 .

[37]  W. Gasarch,et al.  The Book Review Column 1 Coverage Untyped Systems Simple Types Recursive Types Higher-order Systems General Impression 3 Organization, and Contents of the Book , 2022 .

[38]  Jamie B. Coble,et al.  Prognostics and Health Management in Nuclear Power Plants: A Review of Technologies and Applications , 2012 .

[39]  Noureddine Zerhouni,et al.  Remaining Useful Life Estimation of Critical Components With Application to Bearings , 2012, IEEE Transactions on Reliability.

[40]  ChengXiang Zhai,et al.  Instance Weighting for Domain Adaptation in NLP , 2007, ACL.

[41]  C. Bailey,et al.  A Physics-of-failure based Prognostic Method for Power Modules , 2008, 2008 10th Electronics Packaging Technology Conference.

[42]  Raghunathan Rengaswamy,et al.  A review of process fault detection and diagnosis: Part III: Process history based methods , 2003, Comput. Chem. Eng..

[43]  Andre Kleyner,et al.  A New Application for Failure Prognostics – Reduction of Automotive Electronics Reliability Test Duration , 2017 .

[44]  Fabrice Rossi,et al.  Mean Absolute Percentage Error for regression models , 2016, Neurocomputing.

[45]  Tao Zhang,et al.  Deep Model Based Domain Adaptation for Fault Diagnosis , 2017, IEEE Transactions on Industrial Electronics.

[46]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[47]  Ying Peng,et al.  Current status of machine prognostics in condition-based maintenance: a review , 2010 .

[48]  Vladimir Vovk,et al.  A tutorial on conformal prediction , 2007, J. Mach. Learn. Res..

[49]  Joseph Mathew,et al.  Rotating machinery prognostics. State of the art, challenges and opportunities , 2009 .

[50]  Kenneth A. Loparo,et al.  Physically based diagnosis and prognosis of cracked rotor shafts , 2002, SPIE Defense + Commercial Sensing.

[51]  Yaguo Lei,et al.  Deep Convolutional Transfer Learning Network: A New Method for Intelligent Fault Diagnosis of Machines With Unlabeled Data , 2019, IEEE Transactions on Industrial Electronics.

[52]  Brigitte Chebel-Morello,et al.  PRONOSTIA : An experimental platform for bearings accelerated degradation tests. , 2012 .

[53]  Shi Jianming,et al.  A data-driven prognostics approach for RUL based on principle component and instance learning , 2016, 2016 IEEE International Conference on Prognostics and Health Management (ICPHM).

[54]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[55]  Alexander Gammerman,et al.  Prediction algorithms and confidence measures based on algorithmic randomness theory , 2002, Theor. Comput. Sci..

[56]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[57]  Bin Yang,et al.  An intelligent fault diagnosis approach based on transfer learning from laboratory bearings to locomotive bearings , 2019, Mechanical Systems and Signal Processing.

[58]  Kai Goebel,et al.  A Survey of Artificial Intelligence for Prognostics , 2007, AAAI Fall Symposium: Artificial Intelligence for Prognostics.

[59]  Ran Zhang,et al.  Transfer Learning With Neural Networks for Bearing Fault Diagnosis in Changing Working Conditions , 2017, IEEE Access.

[60]  Chetan Gupta,et al.  Long Short-Term Memory Network for Remaining Useful Life estimation , 2017, 2017 IEEE International Conference on Prognostics and Health Management (ICPHM).

[61]  Zhibin Zhao,et al.  Deep Transfer Learning Based on Sparse Autoencoder for Remaining Useful Life Prediction of Tool in Manufacturing , 2019, IEEE Transactions on Industrial Informatics.

[62]  Jianbo Yu,et al.  A similarity-based prognostics approach for Remaining Useful Life estimation of engineered systems , 2008, 2008 International Conference on Prognostics and Health Management.

[63]  Gabriela Medina-Oliva,et al.  Predictive diagnosis based on a fleet-wide ontology approach , 2014, Knowl. Based Syst..

[64]  Benoît Iung,et al.  Remaining useful life estimation based on stochastic deterioration models: A comparative study , 2013, Reliab. Eng. Syst. Saf..

[65]  Abhinav Saxena,et al.  Performance Benchmarking and Analysis of Prognostic Methods for CMAPSS Datasets , 2020, International Journal of Prognostics and Health Management.

[66]  Raghunathan Rengaswamy,et al.  A review of process fault detection and diagnosis: Part I: Quantitative model-based methods , 2003, Comput. Chem. Eng..

[67]  Chetan Gupta,et al.  Framework for Unifying Model-based and Data-driven Fault Diagnosis , 2018, Annual Conference of the PHM Society.

[68]  Xiaoli Li,et al.  Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life , 2016, DASFAA.

[69]  Hongchao Wang,et al.  Data-Driven Methods for Predictive Maintenance of Industrial Equipment: A Survey , 2019, IEEE Systems Journal.

[70]  Slawomir Nowaczyk,et al.  Incorporating Expert Knowledge into a Self-Organized Approach for Predicting Compressor Faults in a City Bus Fleet , 2015, SCAI.

[71]  Raymond J. Mooney,et al.  Mapping and Revising Markov Logic Networks for Transfer Learning , 2007, AAAI.

[72]  M. Shafiee A. Kolios A multi-criteria decision model to mitigate the operational risks of offshore wind infrastructures , 2014 .

[73]  François Laviolette,et al.  Domain-Adversarial Neural Networks , 2014, ArXiv.

[74]  Samuel H. Huang,et al.  System health monitoring and prognostics — a review of current paradigms and practices , 2006 .

[75]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[76]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[77]  Jay Lee,et al.  Review and recent advances in battery health monitoring and prognostics technologies for electric vehicle (EV) safety and mobility , 2014 .

[78]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[79]  Donghua Zhou,et al.  Remaining useful life estimation - A review on the statistical data driven approaches , 2011, Eur. J. Oper. Res..

[80]  Qiang Yang,et al.  Transitive Transfer Learning , 2015, KDD.

[81]  Hamid Reza Karimi,et al.  A review of diagnostics and prognostics of low-speed machinery towards wind turbine farm-level health management , 2016 .

[82]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[83]  Khanh Le Son,et al.  Remaining useful life estimation on the non-homogenous gamma with noise deterioration based on Gibbs filtering: A case study , 2012, 2012 IEEE Conference on Prognostics and Health Management.

[84]  Yiqiang Chen,et al.  Cross-People Mobile-Phone Based Activity Recognition , 2011, IJCAI.

[85]  Rixin Wang,et al.  Cross-Domain Fault Diagnosis Using Knowledge Transfer Strategy: A Review , 2019, IEEE Access.

[86]  Ramesh Nallapati,et al.  A Comparative Study of Methods for Transductive Transfer Learning , 2007, Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007).

[87]  Haibo He,et al.  A Hierarchical Deep Domain Adaptation Approach for Fault Diagnosis of Power Plant Thermal System , 2019, IEEE Transactions on Industrial Informatics.

[88]  Liang Gao,et al.  A New Deep Transfer Learning Based on Sparse Auto-Encoder for Fault Diagnosis , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[89]  Rui Kang,et al.  Experiments for PHM: Needs, developments and challenges , 2014 .

[90]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[91]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[92]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[93]  Ruqiang Yan,et al.  Highly Accurate Machine Fault Diagnosis Using Deep Transfer Learning , 2019, IEEE Transactions on Industrial Informatics.

[94]  Slawomir Nowaczyk,et al.  Evaluation of Self-Organized Approach for Predicting Compressor Faults in a City Bus Fleet , 2015, INNS Conference on Big Data.

[95]  Ö. Eker,et al.  Major challenges in prognostics: study on benchmarking prognostic datasets , 2012 .

[96]  Yu Peng,et al.  A modified echo state network based remaining useful life estimation approach , 2012, 2012 IEEE Conference on Prognostics and Health Management.

[97]  K. Goebel,et al.  Metrics for evaluating performance of prognostic techniques , 2008, 2008 International Conference on Prognostics and Health Management.

[98]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[99]  Houxiang Zhang,et al.  A Comprehensive Survey of Prognostics and Health Management Based on Deep Learning for Autonomous Ships , 2019, IEEE Transactions on Reliability.

[100]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[101]  Leone Giacomo,et al.  A Data-Driven Prognostic Approach Based on Sub-Fleet Knowledge Extraction , 2016 .

[102]  Raghunathan Rengaswamy,et al.  A review of process fault detection and diagnosis: Part II: Qualitative models and search strategies , 2003, Comput. Chem. Eng..

[103]  Qiang Yang,et al.  Distant Domain Transfer Learning , 2017, AAAI.

[104]  Luc Van Gool,et al.  Transferring activities: Updating human behavior analysis , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[105]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[106]  R. Keith Mobley,et al.  An introduction to predictive maintenance , 1989 .

[107]  Jianjun Shi,et al.  A Data-Level Fusion Model for Developing Composite Health Indices for Degradation Modeling and Prognostic Analysis , 2013, IEEE Transactions on Automation Science and Engineering.

[108]  Yuxin Cui,et al.  Transfer Learning with Deep Recurrent Neural Networks for Remaining Useful Life Estimation , 2018, Applied Sciences.