Transient phenomena prediction using recurrent neural networks

To overcome the cost of numerical simulations of transient phenomena, the goal is to construct a robust spatio-temporal reduced model capable of long-term in time predictions. The construction proposed in this article has to deal with several constraints: the reference model is a black box with high dimensional inputs and outputs, long-term in time prediction, few learning samples available, non-linear behaviour and the construction time must remain reasonable while the prediction time must be negligible. Recurrent neural networks are predictive models adapted to this dynamic framework. The improvements of the construction methodology detailed in this paper are the weights optimization through a multilevel optimization approach, a robust construction based on cross-validation and an application of sensitivity analysis in order to reduce the input dimension of the network. Finally, this construction is validated on an industrial test case predicting the temperature of an electronic equipment located in the avionic bay and subjected to fluctuations of its boundary conditions.

[1]  Constantinos C. Pantelides,et al.  Monte Carlo evaluation of derivative-based global sensitivity measures , 2009, Reliab. Eng. Syst. Saf..

[2]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[3]  Guillaume Lecué,et al.  Oracle inequalities for cross-validation type procedures , 2012 .

[4]  Gérard Dreyfus,et al.  Neural networks - methodology and applications , 2005 .

[5]  Jun Li,et al.  Identification of dynamical systems using radial basis function neural networks with hybrid learning algorithm , 2006, 2006 1st International Symposium on Systems and Control in Aerospace and Astronautics.

[6]  Herbert Jaeger,et al.  A tutorial on training recurrent neural networks , covering BPPT , RTRL , EKF and the " echo state network " approach - Semantic Scholar , 2005 .

[7]  Bernard Widrow,et al.  Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[8]  Kurt Tutschku,et al.  Recurrent Multilayer Perceptrons for Identification and Control: The Road to Applications , 1995 .

[9]  Mohammad Bagher Menhaj,et al.  Modelling of Thermal Two Dimensional Free Turbulent Jet by a Three Layer Two Time Scale Cellular Neural Network , 1999, Fuzzy Days.

[10]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[11]  Jus Kocijan,et al.  Dynamical systems identification using Gaussian process models with incorporated local models , 2011, Eng. Appl. Artif. Intell..

[12]  Akio Ushida,et al.  GENERATION OF VARIOUS TYPES OF SPATIO-TEMPORAL PHENOMENA IN TWO-LAYER CELLULAR NEURAL NETWORKS , 2004 .

[13]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[14]  Matieyendou Lamboni,et al.  Derivative-based global sensitivity measures: General links with Sobol' indices and numerical tests , 2012, Math. Comput. Simul..

[15]  Chin-Teng Lin,et al.  Runge-Kutta neural network for identification of dynamical systems in high accuracy , 1998, IEEE Trans. Neural Networks.

[16]  Clemens J. M. Lasance,et al.  Ten Years of Boundary-Condition- Independent Compact Thermal Modeling of Electronic Parts: A Review , 2008 .

[17]  Béatrice Laurent,et al.  Multilayer perceptron for the learning of spatio-temporal dynamics - application in thermal engineering , 2013, Eng. Appl. Artif. Intell..

[18]  A. V. Olgac,et al.  Performance Analysis of Various Activation Functions in Generalized MLP Architectures of Neural Networks , 2011 .

[19]  Barak A. Pearlmutter.  Gradient calculations for dynamic recurrent neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[20]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[21]  Joseph A. C. Delaney.  Sensitivity analysis , 2018, The African Continental Free Trade Area: Economic and Distributional Effects.

[22]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[23]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[24]  Wen Yu,et al.  Training Cellular Neural Networks with Stable Learning Algorithm , 2006, ISNN.

[25]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .