An Improved Grasshopper Optimization Algorithm Based Echo State Network for Predicting Faults in Airplane Engines

In today’s age of industrialization, sensor devices installed on equipment generate a vast amount of data. One of the engineers’ main jobs is utilizing these data to provide better solutions to industrial problems. This availability of extensive data partly led to the creation of predictive maintenance (PdM). In PdM, existing and previous conditions of devices are used to predict their future behavior for optimal maintenance. Most of these PdM approaches are typical time-series predictions. Machine learning tools like Recurrent Neural Networks (RNNs) are excellent tools for time-series predictions. However, most RNNs suffer from training issues due to the unstable gradient problem. Thus, networks such as the Echo State Network (ESN), were designed to solve them. The ESN solves the gradient problem by training only the output weights using simple linear regression. Despite this ease, the selection of ESN parameters and topology is a considerable design challenge. This problem is often formulated as a typical optimization problem. Metaheuristic algorithms are known to be excellent tools for solving optimization problems. Hence, in this work, we design an improved Grasshopper Optimization Algorithm (GOA) based ESN. The proposed technique uses a new solution representation with a simplified attraction and repulsion mechanisms to enhance performance. Our target application is to predict the Remaining Useful Life (RUL) of turbofan engines. The method outperforms the Cuckoo Search (CS), Differential Evolution (DE), Particle Swarm Optimization (PSO), Binary PSO (BPSO), the original GOA, the classical ESN, deep ESN, and LSTM. We have provided all implemented codes and data at the GitHub repository. https://github.com/bala-221/Airplane-fault-prediction

[1]  Emmanuel Ramasso,et al.  Investigating computational geometry for failure prognostics , 2014, International Journal of Prognostics and Health Management.

[2]  Bhaskar Saha,et al.  An Adaptive Recurrent Neural Network for Remaining Useful Life Prediction of Lithium-ion Batteries , 2010 .

[3]  Rosdiazli Ibrahim,et al.  Cuckoo Search Based Optimization of Echo State Network for Time Series Prediction , 2018, 2018 International Conference on Intelligent and Advanced System (ICIAS).

[4]  Min-Yuan Cheng,et al.  Accurately predicting building energy performance using evolutionary multivariate adaptive regression splines , 2014, Appl. Soft Comput..

[5]  Ruqiang Yan,et al.  Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks , 2017, Sensors.

[6]  Adel M. Alimi,et al.  Bi-level multi-objective evolution of a Multi-Layered Echo-State Network Autoencoder for data representations , 2019, Neurocomputing.

[7]  Kwok-wing Chau,et al.  Improving Forecasting Accuracy of Annual Runoff Time Series Using ARIMA Based on EEMD Decomposition , 2015, Water Resources Management.

[8]  Garrison W. Cottrell,et al.  Deep-ESN: A Multiple Projection-encoding Hierarchical Reservoir Computing Framework , 2017, ArXiv.

[9]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[10]  Mantas Lukosevicius,et al.  A Practical Guide to Applying Echo State Networks , 2012, Neural Networks: Tricks of the Trade.

[11]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[12]  Teresa Bernarda Ludermir,et al.  An approach to reservoir computing design and training , 2013, Expert Syst. Appl..

[13]  Stephen J. Simpson,et al.  A behavioural analysis of phase change in the desert locust , 1999 .

[14]  Hossam Faris,et al.  Grasshopper optimization algorithm for multi-objective optimization problems , 2017, Applied Intelligence.

[15]  Sadiq M. Sait,et al.  Applications of Metaheuristics in Reservoir Computing Techniques: A Review , 2018, IEEE Access.

[16]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Sadiq M. Sait,et al.  Iterative computer algorithms with applications in engineering - solving combinatorial optimization problems , 2000 .

[18]  Lin Wang,et al.  Effective electricity energy consumption forecasting using echo state network improved by differential evolution algorithm , 2018 .

[19]  Claudio Gallicchio,et al.  Deep reservoir computing: A critical experimental analysis , 2017, Neurocomputing.

[20]  Martin J. Wainwright,et al.  Divide and Conquer Kernel Ridge Regression , 2013, COLT.

[21]  Fernando José Von Zuben,et al.  An extended echo state network using Volterra filtering and principal component analysis , 2012, Neural Networks.

[22]  Claudio Gallicchio,et al.  Chasing the Echo State Property , 2018, ESANN.

[23]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[24]  Hossam Faris,et al.  Simultaneous Feature Selection and Support Vector Machine Optimization Using the Grasshopper Optimization Algorithm , 2018, Cognitive Computation.

[25]  Frédéric Ferraty,et al.  Functional projection pursuit regression , 2013 .

[26]  Alberto J. Alvares,et al.  Prognostic of RUL based on Echo State Network Optimized by Artificial Bee Colony , 2020 .

[27]  Enrico Zio,et al.  Echo State Network for the Remaining Useful Life Prediction of a Turbofan Engine , 2016, PHM Society European Conference.

[28]  F.O. Heimes,et al.  Recurrent neural networks for remaining useful life estimation , 2008, 2008 International Conference on Prognostics and Health Management.

[29]  Abhinav Saxena,et al.  Damage propagation modeling for aircraft engine run-to-failure simulation , 2008, 2008 International Conference on Prognostics and Health Management.

[30]  Adel M. Alimi,et al.  PSO-based analysis of Echo State Network parameters for time series forecasting , 2017, Appl. Soft Comput..

[31]  Enrico Zio,et al.  Artificial intelligence for fault diagnosis of rotating machinery: A review , 2018, Mechanical Systems and Signal Processing.

[32]  Teresa Bernarda Ludermir,et al.  Genetic algorithm for reservoir computing optimization , 2009, 2009 International Joint Conference on Neural Networks.

[33]  Andrew Lewis,et al.  Grasshopper Optimisation Algorithm: Theory and application , 2017, Adv. Eng. Softw..

[34]  Claudio Gallicchio,et al.  Design of deep echo state networks , 2018, Neural Networks.

[35]  Min Han,et al.  Adaptive Elastic Echo State Network for Multivariate Time Series Prediction , 2016, IEEE Transactions on Cybernetics.

[36]  Rosdiazli Ibrahim,et al.  Prediction Using Cuckoo Search Optimized Echo State Network , 2019, Arabian Journal for Science and Engineering.

[37]  Sule Selcuk,et al.  Predictive maintenance, its implementation and latest trends , 2017 .

[38]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[39]  Li Lin,et al.  Remaining useful life estimation of engineered systems using vanilla LSTM neural networks , 2018, Neurocomputing.

[40]  Xuefeng Yan,et al.  Optimizing the echo state network with a binary particle swarm optimization algorithm , 2015, Knowl. Based Syst..

[41]  Enrique Alba,et al.  An experimental analysis of the Echo State Network initialization using the Particle Swarm Optimization , 2014, 2014 Sixth World Congress on Nature and Biologically Inspired Computing (NaBIC 2014).

[42]  Xiang Li,et al.  Remaining useful life estimation in prognostics using deep convolution neural networks , 2018, Reliab. Eng. Syst. Saf..

[43]  Junfei Qiao,et al.  Adaptive lasso echo state network based on modified Bayesian information criterion for nonlinear system modeling , 2019, Neural Computing and Applications.

[44]  Bin Liang,et al.  Remaining useful life prediction of aircraft engine based on degradation pattern learning , 2017, Reliab. Eng. Syst. Saf..

[45]  Martin J. Wainwright,et al.  Divide and conquer kernel ridge regression: a distributed algorithm with minimax optimal rates , 2013, J. Mach. Learn. Res..

[46]  Lin Lin,et al.  Genetic algorithm optimized double-reservoir echo state network for multi-regime time series prediction , 2017, Neurocomputing.