Deep learning for machine health prognostics using Kernel-based feature transformation

Prognostic health management minimizes system downtime and improves overall equipment effectiveness. Accurate prediction of remaining useful life (RUL) is key to prognostics. Prominent machine learning algorithms implement handcrafted feature extraction to improve RUL prediction. Deep learning automates feature extraction from raw data but requires large datasets and computationally expensive fine-tuning. Data-specific handcrafting and fine-tuning limit the generalization capability of existing models. Proposed framework addresses these challenges using Temporal Multivariate 3D Convolutional Network (TM3C) and Kernel-based Transformation (KT) of features. KT generates 3D features that incorporate trendable degradation patterns from multivariate temporal relationship among sensor data. TM3C implements 3D convolutional layers with temporal filters for RUL prediction. KT is generalizable and improves feature relevance. Full-width filters in TM3C reduce number of tunable parameters and convolution operations. Proposed TM3C-KT capitalizes on the strength of deep learning while lowering the cost for feature discovery, parameter learning, and model fine-tuning. TM3C-KT is evaluated on three prognostics applications, (1) RUL prediction for turbofan engines, (2) Failure state estimation for hydraulic pumps, and (3) Component wear prediction for milling machines. Performance of the framework is comparable and better than benchmark methods in literature. Characteristics of the framework are reviewed on generalizability, prognosability and versatility metrics. Results and corresponding analysis demonstrate suitability of TM3C-KT for industrial applications of machine health prognostics.

[1]  Rafal A. Angryk,et al.  Evaluating Preprocessing Strategies for Time Series Prediction using Deep Learning Architectures , 2017, FLAIRS Conference.

[2]  Abhinav Saxena,et al.  Damage propagation modeling for aircraft engine run-to-failure simulation , 2008, 2008 International Conference on Prognostics and Health Management.

[3]  Wenjing Jin,et al.  Enhanced Restricted Boltzmann Machine With Prognosability Regularization for Prognostics and Health Assessment , 2016, IEEE Transactions on Industrial Electronics.

[4]  Marc G. Genton,et al.  Classes of Kernels for Machine Learning: A Statistics Perspective , 2002, J. Mach. Learn. Res..

[5]  Edward Y. Chang,et al.  Formulating distance functions via the kernel trick , 2005, KDD '05.

[6]  Oliver Schütze,et al.  A Neural Network-Evolutionary Computational Framework for Remaining Useful Life Estimation of Mechanical Systems , 2019, Neural Networks.

[7]  Qun Liu,et al.  DeepSat V2: feature augmented convolutional neural nets for satellite image classification , 2019, Remote Sensing Letters.

[8]  Biqing Huang,et al.  Approach for fault prognosis using recurrent neural network , 2018, J. Intell. Manuf..

[9]  Shuzhi Sam Ge,et al.  Temporal Convolutional Memory Networks for Remaining Useful Life Estimation of Industrial Machinery , 2018, 2019 IEEE International Conference on Industrial Technology (ICIT).

[10]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[11]  Xiaoli Li,et al.  Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life , 2016, DASFAA.

[12]  M. Friedman The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[13]  Takehisa Yairi,et al.  A review on the application of deep learning in system health management , 2018, Mechanical Systems and Signal Processing.

[14]  Nikolai Helwig,et al.  Condition monitoring of a complex hydraulic system using multivariate statistics , 2015, 2015 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) Proceedings.

[15]  Wei Zhang,et al.  Deep learning-based remaining useful life estimation of bearings using multi-scale feature extraction , 2019, Reliab. Eng. Syst. Saf..

[16]  Xiang Li,et al.  Remaining useful life estimation in prognostics using deep convolution neural networks , 2018, Reliab. Eng. Syst. Saf..

[17]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[18]  Taehoon Lee,et al.  Semi-supervised Learning with Deep Generative Models for Asset Failure Prediction , 2017, ArXiv.

[19]  Ruqiang Yan,et al.  Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks , 2017, Sensors.

[20]  Abhinav Saxena,et al.  Review and Analysis of Algorithmic Approaches Developed for Prognostics on CMAPSS Dataset , 2014 .

[21]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[22]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[23]  Jürgen Kurths,et al.  Recurrence plots for the analysis of complex systems , 2009 .

[24]  John Cristian Borges Gamboa,et al.  Deep Learning for Time-Series Analysis , 2017, ArXiv.

[25]  Lin Sun,et al.  Human Action Recognition Using Factorized Spatio-Temporal Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Robert X. Gao,et al.  Deep learning and its applications to machine health monitoring , 2019, Mechanical Systems and Signal Processing.

[27]  Li Lin,et al.  Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network , 2016, 2016 IEEE International Conference on Aircraft Utility Systems (AUS).

[28]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[29]  Jamie B. Coble,et al.  Merging Data Sources to Predict Remaining Useful Life – An Automated Method to Identify Prognostic Parameters , 2010 .

[30]  Minping Jia,et al.  Comprehensive Overview on Computational Intelligence Techniques for Machinery Condition Monitoring and Fault Diagnosis , 2017 .

[31]  Brian A. Weiss,et al.  A review of diagnostic and prognostic capabilities and best practices for manufacturing , 2019, J. Intell. Manuf..

[32]  Kumari Snehkant Lata,et al.  Inferring pathogen-host interactions between Leptospira interrogans and Homo sapiens using network theory , 2019, Scientific Reports.

[33]  Chetan Gupta,et al.  Long Short-Term Memory Network for Remaining Useful Life estimation , 2017, 2017 IEEE International Conference on Prognostics and Health Management (ICPHM).

[34]  Jingtao Lei,et al.  Tool wear predicting based on multi-domain feature fusion by deep convolutional neural network in milling operations , 2019, Journal of Intelligent Manufacturing.

[35]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[36]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[38]  Lovekesh Vig,et al.  Multi-Sensor Prognostics using an Unsupervised Health Index based on LSTM Encoder-Decoder , 2016, ArXiv.

[39]  Tatsuhiko Tsunoda,et al.  DeepInsight: A methodology to transform a non-image data to an image for convolution neural network architecture , 2019, Scientific Reports.

[40]  Yaguo Lei,et al.  Machinery health prognostics: A systematic review from data acquisition to RUL prediction , 2018 .

[41]  Lorenzo Rosasco,et al.  Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review , 2016, International Journal of Automation and Computing.

[42]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[43]  Houxiang Zhang,et al.  Remaining useful life predictions for turbofan engine degradation using semi-supervised deep architecture , 2019, Reliab. Eng. Syst. Saf..

[44]  So Young Sohn,et al.  Multitask learning for health condition identification and remaining useful life prediction: deep convolutional neural network approach , 2020, Journal of Intelligent Manufacturing.

[45]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[46]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[47]  Zhiwu Huang,et al.  Remaining Useful Life Estimation Using CNN-XGB With Extended Time Window , 2019, IEEE Access.

[48]  Samina Khalid,et al.  A survey of feature selection and feature extraction techniques in machine learning , 2014, 2014 Science and Information Conference.

[49]  Michael G. Pecht,et al.  A hybrid prognostics methodology for electronic products , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[50]  Nikolai Helwig,et al.  Automatic feature extraction and selection for classification of cyclical time series data , 2017 .