Fast neural network surrogates for very high dimensional physics-based models in computational oceanography

We present neural network surrogates that provide extremely fast and accurate emulation of a large-scale circulation model for the coupled Columbia River, its estuary and near ocean regions. The circulation model has O(10(7)) degrees of freedom, is highly nonlinear and is driven by ocean, atmospheric and river influences at its boundaries. The surrogates provide accurate emulation of the full circulation code and run over 1000 times faster. Such fast dynamic surrogates will enable significant advances in ensemble forecasts in oceanography and weather.

[1]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[2]  Geoffrey E. Hinton,et al.  Fast Neural Network Emulation of Dynamical Systems for Computer Animation , 1998, NIPS.

[3]  Jose C. Principe,et al.  Prediction of Chaotic Time Series with Neural Networks , 1992 .

[4]  John E. Moody,et al.  The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems , 1991, NIPS.

[5]  P. Grassberger,et al.  NONLINEAR TIME SEQUENCE ANALYSIS , 1991 .

[6]  T. Stein.  International Geoscience And Remote Sensing Symposium , 1992, [Proceedings] IGARSS '92 International Geoscience and Remote Sensing Symposium.

[7]  A. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[8]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[9]  Norman W. Scheffner,et al.  ADCIRC: An Advanced Three-Dimensional Circulation Model for Shelves, Coasts, and Estuaries. Report 1. Theory and Methodology of ADCIRC-2DDI and ADCIRC-3DL. , 1992 .

[10]  Vladimir M. Krasnopolsky,et al.  Some neural network applications in environmental sciences. Part II: advancing computational efficiency of environmental numerical models , 2003, Neural Networks.

[11]  Garrison W. Cottrell,et al.  Non-Linear Dimensionality Reduction , 1992, NIPS.

[12]  Zhengdong Lu,et al.  Sequential Data Assimilation with Sigma-point Kalman Filter on Low-dimensional Manifold June 23 , 2007 DRAFT-DO NOT CIRCULATE , 2007 .

[13]  Alexander F. Shchepetkin,et al.  The regional oceanic modeling system (ROMS): a split-explicit, free-surface, topography-following-coordinate oceanic model , 2005 .

[14]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[15]  Cecelia DeLuca,et al.  Modeling The Earth System , 2003 .

[16]  A. N. Sharkovskiĭ.  Dynamic systems and turbulence , 1989 .

[17]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[18]  Alain Chedin,et al.  A Neural Network Approach for a Fast and Accurate Computation of a Longwave Radiative Budget , 1998 .

[19]  Frank Press,et al.  中国における地震研究(American Geophysical Union,56) , 1977 .

[20]  Nanda Kambhatla,et al.  Dimension Reduction by Local Principal Component Analysis , 1997, Neural Computation.

[21]  Robert M. Farber,et al.  How Neural Nets Work , 1987, NIPS.

[22]  P. Werbos,et al.  Long-term predictions of chemical processes using recurrent neural networks: a parallel training approach , 1992 .

[23]  António M. Baptista,et al.  Author's Personal Copy Dynamics of Atmospheres and Oceans Fast Data Assimilation Using a Nonlinear Kalman Filter and a Model Surrogate: an Application to the Columbia River Estuary , 2022 .

[24]  I. Jolliffe.  Principal Component Analysis , 2002 .

[25]  Vladimir M. Krasnopolsky,et al.  Neural network approximations for nonlinear interactions in wind wave spectra: direct mapping for wind seas in deep water , 2005 .

[26]  William W. Hsieh,et al.  ENSO simulation and prediction in a hybrid coupled model with data assimilation , 2003 .

[27]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[28]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[29]  António M. Baptista,et al.  A cross-scale model for 3D baroclinic circulation in estuary–plume–shelf systems: II. Application to the Columbia River , 2005 .

[30]  Günter Dietrich,et al.  General oceanography: An introduction , 1980 .

[31]  António M. Baptista,et al.  Coastal and estuarine forecast systems. A multi-purpose infrastructure for the Columbia River , 1999 .

[32]  Martin Fodslette Meiller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1993 .

[33]  Rich Caruana,et al.  Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping , 2000, NIPS.

[34]  A. Bennett,et al.  Inverse Modeling of the Ocean and Atmosphere , 2002 .

[35]  Rudolph van der Merwe,et al.  Sigma-point kalman filters for probabilistic inference in dynamic state-space models , 2004 .

[36]  David A. Jay,et al.  Interaction of fluctuating river flow with a barotropic tide: A demonstration of wavelet tidal analysis methods , 1997 .

[37]  Adrian E. Raftery,et al.  Weather Forecasting with Ensemble Methods , 2005, Science.

[38]  Sergey Maksimovich Frolov,et al.  Assimilating in-situ Measurements into a Reduced-Dimensionality Model of an Estuary- Plume System. , 2006 .

[39]  Daniel R. Lynch,et al.  Comprehensive coastal circulation model with application to the Gulf of Maine , 1996 .

[40]  S. Amari,et al.  Network Information Criterion | Determining the Number of Hidden Units for an Articial Neural Network Model Network Information Criterion | Determining the Number of Hidden Units for an Articial Neural Network Model , 2007 .

[41]  C. Lee Giles,et al.  Learning Chaotic Attractors by Neural Networks , 2000, Neural Computation.

[42]  Vladimir M. Krasnopolsky,et al.  A neural network technique to improve computational efficiency of numerical oceanic models , 2002 .

[43]  Edward P. Myers,et al.  A cross-scale model for 3D baroclinic circulation in estuary-plume-shelf systems: I , 2004 .

[44]  R. Ferraro,et al.  Modeling the Earth system. Critical computational technologies that enable us to predict our planet's future , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[45]  William W. Hsieh,et al.  Hybrid coupled modeling of the tropical Pacific using neural networks , 2005 .

[46]  Chris M. Bishop,et al.  Real-Time Control of a Tokamak Plasma Using Neural Networks , 1995, Neural Computation.

[47]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[48]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[49]  Martin Casdagli,et al.  Nonlinear prediction of chaotic time series , 1989 .

[50]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[51]  Gene H. Golub,et al.  Matrix computations , 1983 .