Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics

In numerous industrial applications where safety, efficiency, and reliability are among primary concerns, condition-based maintenance (CBM) is often the most effective and reliable maintenance policy. Prognostics, as one of the key enablers of CBM, involves the core task of estimating the remaining useful life (RUL) of the system. Neural networks-based approaches have produced promising results on RUL estimation, although their performances are influenced by handcrafted features and manually specified parameters. In this paper, we propose a multiobjective deep belief networks ensemble (MODBNE) method. MODBNE employs a multiobjective evolutionary algorithm integrated with the traditional DBN training technique to evolve multiple DBNs simultaneously subject to accuracy and diversity as two conflicting objectives. The eventually evolved DBNs are combined to establish an ensemble model used for RUL estimation, where combination weights are optimized via a single-objective differential evolution algorithm using a task-oriented objective function. We evaluate the proposed method on several prognostic benchmarking data sets and also compare it with some existing approaches. Experimental results demonstrate the superiority of our proposed method.

[1]  J. Orbach.  Principles of Neurodynamics. Perceptrons and the Theory of Brain Mechanisms. , 1962 .

[2]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[4]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[5]  Thomas G. Dietterich.  Machine-Learning Research , 1997, AI Mag..

[6]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[7]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[9]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[10]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[11]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[12]  Hussein A. Abbass,et al.  A Memetic Pareto Evolutionary Approach to Artificial Neural Networks , 2001, Australian Joint Conference on Artificial Intelligence.

[13]  Hussein A. Abbass,et al.  Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[14]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[15]  Gregory L. Plett,et al.  Extended Kalman filtering for battery management systems of LiPB-based HEV battery packs: Part 3. State and parameter estimation , 2004 .

[16]  Nagi Gebraeel,et al.  Residual life predictions from vibration-based degradation signals: a neural network approach , 2004, IEEE Transactions on Industrial Electronics.

[17]  Gregory L. Plett,et al.  Extended Kalman filtering for battery management systems of LiPB-based HEV battery packs: Part 2. Modeling and identification , 2004 .

[18]  Mark Schwabacher,et al.  A Survey of Data -Driven Prognostics , 2005 .

[19]  Xin Yao,et al.  Diversity creation methods: a survey and categorisation , 2004, Inf. Fusion.

[20]  Jing J. Liang,et al.  Performance Evaluation of Multiagent Genetic Algorithm , 2006, Natural Computing.

[21]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[22]  Xin Yao,et al.  Ensemble Learning Using Multi-Objective Evolutionary Algorithms , 2006, J. Math. Model. Algorithms.

[23]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[24]  Lifeng Xi,et al.  Residual life predictions for ball bearings based on self-organizing map and back propagation neural network methods , 2007 .

[25]  J. Wesley Hines,et al.  An Integrated Fuzzy Inference-based Monitoring, Diagnostic, and Prognostic System for Intelligent Control and Maintenance , 2007 .

[26]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[27]  Abhinav Saxena,et al.  Damage propagation modeling for aircraft engine run-to-failure simulation , 2008, 2008 International Conference on Prognostics and Health Management.

[28]  F.O. Heimes,et al.  Recurrent neural networks for remaining useful life estimation , 2008, 2008 International Conference on Prognostics and Health Management.

[29]  J.W. Hines,et al.  Prognostic algorithm categorization with PHM Challenge application , 2008, 2008 International Conference on Prognostics and Health Management.

[30]  L. Peel,et al.  Data driven prognostics using a Kalman filter ensemble of neural network models , 2008, 2008 International Conference on Prognostics and Health Management.

[31]  K. Goebel,et al.  Metrics for evaluating performance of prognostic techniques , 2008, 2008 International Conference on Prognostics and Health Management.

[32]  Keith Worden,et al.  Uncertainty analysis of a neural network used for fatigue lifetime prediction , 2008 .

[33]  Jianbo Yu,et al.  A similarity-based prognostics approach for Remaining Useful Life estimation of engineered systems , 2008, 2008 International Conference on Prognostics and Health Management.

[34]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[35]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[36]  Michael Pecht,et al.  Physics-of-failure-based prognostics for electronic products , 2009 .

[37]  Joseph Mathew,et al.  Rotating machinery prognostics. State of the art, challenges and opportunities , 2009 .

[38]  Kai Goebel,et al.  Comparison of prognostic algorithms for estimating remaining useful life of batteries , 2009 .

[39]  Qingfu Zhang,et al.  Multiobjective Optimization Problems With Complicated Pareto Sets, MOEA/D and NSGA-II , 2009, IEEE Transactions on Evolutionary Computation.

[40]  Gang Niu,et al.  Development of an optimized condition-based maintenance system by data fusion and reliability-centered maintenance , 2010, Reliab. Eng. Syst. Saf..

[41]  Hatem M. Elattar,et al.  Evaluation of Neural Networks in the Subject of Prognostics As Compared To Linear Regression Model , 2010 .

[42]  Zhigang Tian,et al.  A neural network approach for remaining useful life prediction utilizing both failure and suspension histories , 2010 .

[43]  Enrico Zio,et al.  A data-driven fuzzy approach for predicting the remaining useful life in dynamic failure scenarios of a nuclear system , 2010, Reliab. Eng. Syst. Saf..

[44]  Huanhuan Chen,et al.  Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[45]  Sankalita Saha,et al.  Distributed prognostic health management with gaussian process regression , 2010, 2010 IEEE Aerospace Conference.

[46]  Bin Zhang,et al.  Machine Condition Prediction Based on Adaptive Neuro–Fuzzy and High-Order Particle Filtering , 2011, IEEE Transactions on Industrial Electronics.

[47]  Chao Hu,et al.  Ensemble of data-driven prognostic algorithms for robust prediction of remaining useful life , 2011, 2011 IEEE Conference on Prognostics and Health Management.

[48]  Tara N. Sainath,et al.  Deep Belief Networks using discriminative features for phone recognition , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[49]  Byeng D. Youn,et al.  Resilience-Driven System Design of Complex Engineered Systems , 2011, DAC 2011.

[50]  Byeng D. Youn,et al.  A generic probabilistic framework for structural health prognostics and uncertainty management , 2012 .

[51]  Taejin Kim,et al.  Semi-supervised learning with co-training for data-driven prognostics , 2012 .

[52]  A. Kai Qin,et al.  An improved CUDA-based implementation of differential evolution on GPU , 2012, GECCO '12.

[53]  Zheng Hu,et al.  A Technical Framework and Roadmap of Embedded Diagnostics and Prognostics for Complex Mechanical Systems in Prognostics and Health Management Systems , 2012, IEEE Transactions on Reliability.

[54]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[55]  Pingfeng Wang,et al.  Deep Belief Network based state classification for structural health diagnosis , 2012, 2012 IEEE Aerospace Conference.

[56]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[57]  Shengli Wu,et al.  Effective Neural Network Ensemble Approach for Improving Generalization Performance , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[58]  Pedro Antonio Gutiérrez,et al.  Negative Correlation Ensemble Learning for Ordinal Regression , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[59]  Itamar Arel,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1 Ensemble Learning in Fixed Expansion Layer Network , 2022 .

[60]  Xiaodong Li,et al.  Differential evolution on the CEC-2013 single-objective continuous optimization testbed , 2013, 2013 IEEE Congress on Evolutionary Computation.

[61]  Pingfeng Wang,et al.  Failure diagnosis using deep belief learning based health state classification , 2013, Reliab. Eng. Syst. Saf..

[62]  Cairo Lucio Nascimento Junior,et al.  GFRBS-PHM: A Genetic Fuzzy Rule-Based System for PHM with improved interpretability , 2013, 2013 IEEE Conference on Prognostics and Health Management (PHM).

[63]  Enrico Zio,et al.  Ensemble neural network-based particle filtering for prognostics , 2013 .

[64]  Enrico Zio,et al.  Predicting component reliability and level of degradation with complex-valued neural networks , 2014, Reliab. Eng. Syst. Saf..

[65]  Brendan McCane,et al.  Deep Networks are Effective Encoders of Periodicity , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[66]  Kay Chen Tan,et al.  Estimation of Remaining Useful Life Based on Switching Kalman Filter Neural Network Ensemble , 2014 .

[67]  Kunikazu Kobayashi,et al.  Time series forecasting using a deep belief network with restricted Boltzmann machines , 2014, Neurocomputing.

[68]  Jiuping Xu,et al.  PHM-Oriented Integrated Fusion Prognostics for Aircraft Engines Based on Sensor Data , 2014, IEEE Sensors Journal.

[69]  Abhinav Saxena,et al.  Review and Analysis of Algorithmic Approaches Developed for Prognostics on CMAPSS Dataset , 2014 .

[70]  Robert F. Harrison,et al.  Fusing an ensemble of diverse prognostic life predictions , 2014, 2014 IEEE Aerospace Conference.

[71]  Wenhua Zeng,et al.  A New Local Search-Based Multiobjective Optimization Algorithm , 2015, IEEE Transactions on Evolutionary Computation.

[72]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[73]  Noureddine Zerhouni,et al.  A New Multivariate Approach for Prognostics Based on Extreme Learning Machine and Fuzzy Clustering , 2015, IEEE Transactions on Cybernetics.

[74]  Ye Tian,et al.  An Efficient Approach to Nondominated Sorting for Evolutionary Multiobjective Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[75]  Kay Chen Tan,et al.  Deep Belief Networks Ensemble with Multi-objective Optimization for Failure Diagnosis , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[76]  Kay Chen Tan,et al.  Online Diversity Assessment in Evolutionary Multiobjective Optimization: A Geometrical Perspective , 2015, IEEE Transactions on Evolutionary Computation.

[77]  Hussein A. Abbass,et al.  Adaptive Cross-Generation Differential Evolution Operators for Multiobjective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[78]  Haizhou Li,et al.  How the Brain Formulates Memory: A Spatio-Temporal Model Research Frontier , 2016, IEEE Computational Intelligence Magazine.

[79]  Robert G. Reynolds,et al.  Leveraged Neighborhood Restructuring in Cultural Algorithms for Solving Real-World Numerical Optimization Problems , 2016, IEEE Transactions on Evolutionary Computation.

[80]  Bo Zhang,et al.  Balancing Convergence and Diversity in Decomposition-Based Many-Objective Optimizers , 2016, IEEE Transactions on Evolutionary Computation.

[81]  Fang Liu,et al.  A Multiobjective Evolutionary Algorithm Based on Decision Variable Analyses for Multiobjective Optimization Problems With Large-Scale Variables , 2016, IEEE Transactions on Evolutionary Computation.

[82]  Kay Chen Tan,et al.  Multimodal Degradation Prognostics Based on Switching Kalman Filter Ensemble , 2017, IEEE Transactions on Neural Networks and Learning Systems.