Linear theory for filtering nonlinear multiscale systems with model error

In this paper, we study filtering of multiscale dynamical systems with model error arising from limitations in resolving the smaller scale processes. In particular, the analysis assumes the availability of continuous-time noisy observations of all components of the slow variables. Mathematically, this paper presents new results on higher order asymptotic expansion of the first two moments of a conditional measure. In particular, we are interested in the application of filtering multiscale problems in which the conditional distribution is defined over the slow variables, given noisy observation of the slow variables alone. From the mathematical analysis, we learn that for a continuous time linear model with Gaussian noise, there exists a unique choice of parameters in a linear reduced model for the slow variables which gives the optimal filtering when only the slow variables are observed. Moreover, these parameters simultaneously give the optimal equilibrium statistical estimates of the underlying system, and as a consequence they can be estimated offline from the equilibrium statistics of the true signal. By examining a nonlinear test model, we show that the linear theory extends in this non-Gaussian, nonlinear configuration as long as we know the optimal stochastic parametrization and the correct observation model. However, when the stochastic parametrization model is inappropriate, parameters chosen for good filter performance may give poor equilibrium statistical estimates and vice versa; this finding is based on analytical and numerical results on our nonlinear test model and the two-layer Lorenz-96 model. Finally, even when the correct stochastic ansatz is given, it is imperative to estimate the parameters simultaneously and to account for the nonlinear feedback of the stochastic parameters into the reduced filter estimates. In numerical experiments on the two-layer Lorenz-96 model, we find that the parameters estimated online, as part of a filtering procedure, simultaneously produce accurate filtering and equilibrium statistical prediction. In contrast, an offline estimation technique based on a linear regression, which fits the parameters to a training dataset without using the filter, yields filter estimates which are worse than the observations or even divergent when the slow variables are not fully observed. This finding does not imply that all offline methods are inherently inferior to the online method for nonlinear estimation problems, it only suggests that an ideal estimation technique should estimate all parameters simultaneously whether it is online or offline.

[1]  Andrew J. Majda,et al.  An ensemble Kalman filter for statistical estimation of physics constrained nonlinear regression models , 2014, J. Comput. Phys..

[2]  Andrew J. Majda,et al.  Quantifying Bayesian filter performance for turbulent dynamical systems through information theory , 2014 .

[3]  Istvan Szunyogh,et al.  Local ensemble Kalman filtering in the presence of model bias , 2006 .

[4]  Michael Ghil,et al.  An efficient algorithm for estimating noise covariances in distributed systems , 1985 .

[5]  Andrew J. Majda,et al.  Physics constrained nonlinear regression models for time series , 2012 .

[6]  J. Harlim.  Data Assimilation with Model Error from Unresolved Scales , 2013 .

[7]  J. Yukich,et al.  Limit theory for point processes in manifolds , 2011, 1104.0914.

[8]  E. Kalnay,et al.  C ○ 2007 The Authors , 2006 .

[9]  Istvan Szunyogh,et al.  A Local Ensemble Kalman Filter for Atmospheric Data Assimilation , 2002 .

[10]  Andrew J. Majda,et al.  Lessons in uncertainty quantification for turbulent dynamical systems , 2012 .

[11]  I. Moroz,et al.  Stochastic parametrizations and model uncertainty in the Lorenz ’96 system , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[12]  R. Mehra.  On the identification of variances and adaptive Kalman filtering , 1970 .

[13]  T. Sauer,et al.  Adaptive ensemble Kalman filtering of non-linear systems , 2013 .

[14]  Andrew J. Majda,et al.  Fundamental limitations of polynomial chaos for uncertainty quantification in systems with intermittent instabilities , 2013 .

[15]  Andrew M. Stuart,et al.  Evaluating Data Assimilation Algorithms , 2011, ArXiv.

[16]  Arlindo da Silva,et al.  Data assimilation in the presence of forecast bias , 1998 .

[17]  T. Miyoshi.  The Gaussian Approach to Adaptive Covariance Inflation and Its Implementation with the Local Ensemble Transform Kalman Filter , 2011 .

[18]  Jeffrey L. Anderson,et al.  An adaptive covariance inflation error correction algorithm for ensemble filters , 2007 .

[19]  A. Majda,et al.  Normal forms for reduced stochastic climate models , 2009, Proceedings of the National Academy of Sciences.

[20]  Andrew J. Majda,et al.  Quantifying uncertainty for predictions with model error in non-Gaussian systems with intermittency , 2012 .

[21]  James A. Hansen,et al.  Efficient Approximate Techniques for Integrating Stochastic Differential Equations , 2006 .

[22]  P. Imkeller,et al.  Dimensional reduction in nonlinear filtering: A homogenization approach , 2011, 1112.2986.

[23]  H. Kushner.  On the Differential Equations Satisfied by Conditional Probablitity Densities of Markov Processes, with Applications , 1964 .

[24]  D. Wilks.  Effects of stochastic parametrizations in the Lorenz '96 system , 2005 .

[25]  Craig H. Bishop,et al.  Adaptive sampling with the ensemble transform Kalman filter , 2001 .

[26]  Eric Vanden-Eijnden,et al.  A computational strategy for multiscale systems with applications to Lorenz 96 model , 2004 .

[27]  Andrew J. Majda,et al.  Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation , 2010, J. Comput. Phys..

[28]  Andrew J. Majda,et al.  Mathematical strategies for filtering complex systems: Regularly spaced sparse observations , 2008, J. Comput. Phys..

[29]  Eric Vanden-Eijnden,et al.  Subgrid-Scale Parameterization with Conditional Markov Chains , 2008 .

[30]  Andrew J Majda,et al.  Quantifying uncertainty in climate change science through empirical information theory , 2010, Proceedings of the National Academy of Sciences.

[31]  Andrew J. Majda,et al.  A mathematical framework for stochastic climate models , 2001 .

[32]  Andrew J. Majda,et al.  Test models for improving filtering with model errors through stochastic parameter estimation , 2010, J. Comput. Phys..

[33]  E. Kalnay,et al.  Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter , 2009 .

[34]  Hong Li,et al.  Data Assimilation as Synchronization of Truth and Model: Experiments with the Three-Variable Lorenz System* , 2006 .

[35]  Georg A. Gottwald,et al.  The role of additive and multiplicative noise in filtering complex dynamical systems , 2013, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[36]  D. Crisan,et al.  Fundamentals of Stochastic Filtering , 2008 .

[37]  Andrew J Majda,et al.  Improving model fidelity and sensitivity for complex systems through empirical information theory , 2011, Proceedings of the National Academy of Sciences.

[38]  K. Vahala.  Handbook of stochastic methods for physics, chemistry and the natural sciences , 1986, IEEE Journal of Quantum Electronics.

[39]  B. Friedland.  Treatment of bias in recursive filtering , 1969 .

[40]  Raman K. Mehra,et al.  Approaches to adaptive filtering , 1970 .

[41]  Emily L. Kang,et al.  Filtering Partially Observed Multiscale Systems with Heterogeneous Multiscale Methods-Based Reduced Climate Models , 2012 .

[42]  Andrew J. Majda,et al.  A NONLINEAR TEST MODEL FOR FILTERING SLOW-FAST SYSTEMS ∗ , 2008 .

[43]  Andrew J. Majda,et al.  Mathematical strategies for filtering turbulent dynamical systems , 2010 .

[44]  Georg A. Gottwald,et al.  Data Assimilation in Slow–Fast Systems Using Homogenized Climate Models , 2011, 1110.6671.

[45]  Andrew J. Majda,et al.  Filtering skill for turbulent signals for a suite of nonlinear and linear extended Kalman filters , 2012, J. Comput. Phys..

[46]  Fan Zhang.  Parameter estimation and model fitting of stochastic processes , 2011 .

[47]  Andrew J. Majda,et al.  Filtering Complex Turbulent Systems , 2012 .

[48]  P. Bélanger.  Estimation of noise covariance matrices for a linear time-varying stochastic process , 1972, Autom..

[49]  Andrew J. Majda,et al.  Filtering a nonlinear slow-fast system with strong fast forcing , 2010 .

[50]  Jeffrey L. Anderson,et al.  A Monte Carlo Implementation of the Nonlinear Filtering Problem to Produce Ensemble Assimilations and Forecasts , 1999 .

[51]  Thomas M. Hamill,et al.  Ensemble Data Assimilation with the NCEP Global Forecast System , 2008 .

[52]  Andrew J Majda,et al.  Link between statistical equilibrium fidelity and forecasting skill for complex systems with model error , 2011, Proceedings of the National Academy of Sciences.

[53]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[54]  Andrew J. Majda,et al.  Low-Frequency Climate Response and Fluctuation–Dissipation Theorems: Theory and Practice , 2010 .

[55]  B. Øksendal.  Stochastic differential equations : an introduction with applications , 1987 .