Neural Network Emulation of Spatio-temporal Data Using Linear and Nonlinear Dimensionality Reduction

A statistical emulator of a high-fidelity computer model is based on the application of machine learning algorithms to input-output data generated by the model at selected design points. Applications include real-time control, design optimization and inverse parameter estimation. In many of these applications, the outputs are spatial or spatio-temporal fields. In such cases, standard emulation methods are computationally impractical due to the curse of dimensionality, or are limited in their applicability by simplifying assumptions in relation to the correlation structure. In this work, we combine linear and nonlinear dimensionality reduction with artificial neural networks to develop an efficient approach to emulating high-dimensional spatio-temporal models, without making ad hoc assumptions regarding correlations. The approach is tested on models of electromagnetic wave propagation. The necessity of nonlinear dimensionality reduction is highlighted.

[1]  Jeremy E. Oakley,et al.  Multivariate Gaussian Process Emulators With Nonseparable Covariance Structures , 2013, Technometrics.

[2]  C. F. Sirmans,et al.  Nonstationary multivariate process modeling through spatially varying coregionalization , 2004 .

[3]  M. J. Bayarri,et al.  Computer model validation with functional output , 2007, 0711.3271.

[4]  A. O'Hagan,et al.  Bayesian inference for the uncertainty distribution of computer model outputs , 2002 .

[5]  D. Higdon,et al.  Computer Model Calibration Using High-Dimensional Output , 2008 .

[6]  Martin T. Hagan,et al.  Gauss-Newton approximation to Bayesian learning , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[7]  Allen Tannenbaum,et al.  Statistical shape analysis using kernel PCA , 2006, Electronic Imaging.

[8]  Christopher K. I. Williams.  On a Connection between Kernel PCA and Metric Multidimensional Scaling , 2004, Machine Learning.

[9]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[10]  Baskar Ganapathysubramanian,et al.  A non-linear dimension reduction methodology for generating data-driven stochastic input models , 2008, J. Comput. Phys..

[11]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[12]  J. Rougier.  Efficient Emulators for Multivariate Deterministic Functions , 2008 .

[13]  Max D. Morris.  The Design and Analysis of Computer Experiments. Thomas J. Santner , Brian J. Williams , and William I. Notz , 2004 .

[14]  W. Torgerson.  Multidimensional scaling: I. Theory and method , 1952 .

[15]  A. O'Hagan,et al.  Bayesian emulation of complex multi-output and dynamic computer models , 2010 .

[16]  Xin Sun,et al.  Bayesian Treed Multivariate Gaussian Process With Adaptive Design: Application to a Carbon Capture Unit , 2014, Technometrics.

[17]  John Moody,et al.  Prediction Risk and Architecture Selection for Neural Networks , 1994 .

[18]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[19]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.