Predicting time series of railway speed restrictions with time-dependent machine learning techniques

In this paper, a hybrid approach to combine conditional restricted Boltzmann machines (CRBM) and echo state networks (ESN) for binary time series prediction is proposed. Both methods have demonstrated their ability to extract complex dynamic patterns from time-dependent data in several applications and benchmark studies. To the authors' knowledge, it is the first time that the proposed combination of algorithms is applied for reliability prediction. The proposed approach is verified on a case study predicting the occurrence of railway operation disruptions based on discrete-event data, which is represented by a binary time series. The case study concerns speed restrictions affecting railway operations, caused by failures of tilting systems of railway vehicles. The overall prediction accuracy of the algorithm is 99.93%; the prediction accuracy for occurrence of speed restrictions within the foresight period is 98% (which corresponds to the sensitivity of the algorithm). The prediction results of the case study are compared to the prediction with a MLP trained with a Newton conjugate gradient algorithm. The proposed approach proves to be superior to MLP.

[1]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[2]  Frank L. Lewis,et al.  Intelligent Fault Diagnosis and Prognosis for Engineering Systems , 2006 .

[3]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[4]  Geoffrey E. Hinton,et al.  Modeling pigeon behavior using a Conditional Restricted Boltzmann Machine , 2009, ESANN.

[5]  Yoshua Bengio,et al.  Gradient-Based Optimization of Hyperparameters , 2000, Neural Computation.

[6]  Herbert Jaeger,et al.  A tutorial on training recurrent neural networks , covering BPPT , RTRL , EKF and the " echo state network " approach - Semantic Scholar , 2005 .

[7]  Geoffrey E. Hinton,et al.  Factored conditional restricted Boltzmann Machines for modeling motion style , 2009, ICML '09.

[8]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[9]  Yanqing Zhang,et al.  Statistical fuzzy interval neural networks for currency exchange rate time series prediction , 2007, Appl. Soft Comput..

[10]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[11]  David Verstraeten.  Reservoir Computing: computation with dynamical systems , 2009 .

[12]  D. Cox.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[13]  John A. Nelder,et al.  Generalized linear models. 2nd ed. , 1993 .

[14]  Snehamoy Chatterjee,et al.  Reliability estimation using a genetic algorithm-based artificial neural network: An application to a load-haul-dump machine , 2012, Expert Syst. Appl..

[15]  I. Jolliffe.  Principal Component Analysis , 2002 .

[16]  Lain L. MacDonald,et al.  Hidden Markov and Other Models for Discrete- valued Time Series , 1997 .

[17]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[18]  Farid U. Dowla,et al.  Backpropagation Learning for Multilayer Feed-Forward Neural Networks Using the Conjugate Gradient Method , 1991, Int. J. Neural Syst..

[19]  Scott L. Zeger,et al.  A Class of Logistic Regression Models for Multivariate Binary Time Series , 1989 .

[20]  Ping-Feng Pai,et al.  System reliability forecasting by support vector machines with genetic algorithms , 2006, Math. Comput. Model..

[21]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[22]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[23]  B. Silverman,et al.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[24]  Mantas Lukosevicius,et al.  A Practical Guide to Applying Echo State Networks , 2012, Neural Networks: Tricks of the Trade.

[25]  Teresa Bernarda Ludermir,et al.  An approach to reservoir computing design and training , 2013, Expert Syst. Appl..

[26]  Olga Fink.  Predicting potential railway operations disruptions caused by critical component failure using echo state neural networks and automatically collected diagnostic data , 2013 .

[27]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[28]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[29]  Robert Dorbritz.  Methodology for assessing the structural and operational robustness of railway networks , 2012 .

[30]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[31]  Zarita Zainuddin,et al.  Modified wavelet neural network in function approximation and its application in prediction of time-series pollution data , 2011, Appl. Soft Comput..

[32]  S. P. Pederson,et al.  Hidden Markov and Other Models for Discrete-Valued Time Series , 1998 .

[33]  Prem Kumar Kalra,et al.  Time-series prediction with single integrate-and-fire neuron , 2007, Appl. Soft Comput..

[34]  Bonnie K. Ray,et al.  Regression Models for Time Series Analysis , 2003, Technometrics.

[35]  Loon Ching Tang,et al.  Application of neural networks in forecasting engine systems reliability , 2003, Appl. Soft Comput..

[36]  Guang-Bin Huang,et al.  Learning capability and storage capacity of two-hidden-layer feedforward networks , 2003, IEEE Trans. Neural Networks.

[37]  Zehong Yang,et al.  Short-term stock price prediction based on echo state networks , 2009, Expert Syst. Appl..