Lagrangian uncertainty quantification and information inequalities for stochastic flows

We develop a systematic information-theoretic framework for quantification and mitigation of error in probabilistic Lagrangian (i.e., trajectory-based) predictions which are obtained from (Eulerian) vector fields generating the underlying dynamical system in a way which naturally applies in both deterministic and stochastic settings. This work is motivated by the desire to improve Lagrangian predictions in complex, multi-scale systems based on simplified, data-driven models. Here, discrepancies between probability measures $\mu$ and $\nu$ associated with the true dynamics and its approximation are quantified via so-called $\varphi$-divergencies, $\mathcal{D}_\varphi(\mu\|\nu)$, which are premetrics defined by a class of strictly convex functions $\varphi$. We derive general information bounds on the uncertainty in estimates, $\mathbb{E}^{\nu}[f]$, of `true' observables $\mathbb{E}^{\mu}[f]$ in terms of $\varphi$-divergencies; we then derive two distinct bounds on $\mathcal{D}_\varphi(\mu\|\nu)$ itself. First, an analytically tractable bound on $\mathcal{D}_\varphi(\mu\|\nu)$ is derived from differences between vector fields generating the true dynamics and its approximations. The second bound on $\mathcal{D}_\varphi(\mu\|\nu)$ is based on a difference of so-called finite-time divergence rate (FTDR) fields and it can be exploited within a computational framework to mitigate the error in Lagrangian predictions by tuning the fields of expansion rates obtained from simplified models. This new framework provides a systematic link between Eulerian (field-based) model error and the resulting uncertainty in Lagrangian (trajectory-based) predictions.

[1]  Andrew J Majda,et al.  Improving model fidelity and sensitivity for complex systems through empirical information theory , 2011, Proceedings of the National Academy of Sciences.

[2]  B. Harris Parametric Statistical Models and Likelihood (Lecture Notes in Statistics 50) , 1991 .

[3]  H. Kunita Stochastic differential equations and stochastic flows of diffeomorphisms , 1984 .

[4]  Shun-ichi Amari,et al.  $\alpha$ -Divergence Is Unique, Belonging to Both $f$-Divergence and Bregman Divergence Classes , 2009, IEEE Transactions on Information Theory.

[5]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[6]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[7]  Alessio Figalli,et al.  Existence and uniqueness of martingale solutions for SDEs with rough or degenerate coefficients , 2008 .

[8]  Igor Vajda,et al.  On Divergences and Informations in Statistics and Information Theory , 2006, IEEE Transactions on Information Theory.

[9]  Giuseppe Da Prato,et al.  Introduction to Stochastic Analysis and Malliavin Calculus , 2007 .

[10]  D. G. Chapman,et al.  Minimum Variance Estimation Without Regularity Assumptions , 1951 .

[11]  N. Čencov Statistical Decision Rules and Optimal Inference , 2000 .

[12]  Eric Vanden-Eijnden,et al.  Large Deviations in Fast–Slow Systems , 2015, 1510.02227.

[13]  I. Csiszár Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[14]  L. Ambrosio,et al.  Gradient Flows: In Metric Spaces and in the Space of Probability Measures , 2005 .

[15]  S. Varadhan,et al.  Asymptotic evaluation of certain Markov process expectations for large time , 1975 .

[16]  L. Arnold Random Dynamical Systems , 2003 .

[17]  K. Elworthy,et al.  Formulae for the Derivatives of Heat Semigroups , 1994, 1911.10971.

[18]  L. Bregman The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[19]  D. Nualart The Malliavin Calculus and Related Topics , 1995 .

[20]  K. Elworthy Stochastic Flows on Riemannian Manifolds , 1992 .

[21]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[22]  Yannis Pantazis,et al.  Sensitivity analysis for rare events based on Rényi divergence , 2018, The Annals of Applied Probability.

[23]  David R. Anderson,et al.  Model Selection and Multimodel Inference , 2003 .

[24]  S. Wiggins,et al.  Transport in two-dimensional maps , 1990 .

[25]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[26]  Paul Dupuis,et al.  Distinguishing and integrating aleatoric and epistemic variation in uncertainty quantification , 2011, 1103.1861.

[27]  Grigorios A. Pavliotis,et al.  Multiscale Methods: Averaging and Homogenization , 2008 .

[28]  Kenneth Uda,et al.  Path-based measures of expansion rates and Lagrangian transport in stochastic flows , 2018, ArXiv.

[29]  Jing Li,et al.  Computation of Failure Probability Subject to Epistemic Uncertainty , 2012, SIAM J. Sci. Comput..

[30]  M. Freidlin,et al.  Random Perturbations of Dynamical Systems , 1984 .

[31]  Yuri Kifer,et al.  L2 Diffusion Approximation for Slow Motion in Averaging , 2003 .

[32]  Andrew J Majda,et al.  Link between statistical equilibrium fidelity and forecasting skill for complex systems with model error , 2011, Proceedings of the National Academy of Sciences.

[33]  O. Barndorff-Nielsen Information and Exponential Families in Statistical Theory , 1980 .

[34]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[35]  C. Léonard Minimization of Energy Functionals Applied to Some Inverse Problems , 2001 .

[36]  M. Yor,et al.  Continuous martingales and Brownian motion , 1990 .

[37]  S. Amari,et al.  Information geometry of divergence functions , 2010 .

[38]  P. Cattiaux,et al.  Minimization of the Kullback information of diffusion processes , 1994 .

[39]  Absolute continuity/singularity and relative entropy properties for probability measures induced by diffusions on infinite time intervals , 2005 .

[40]  G. Haller Distinguished material surfaces and coherent structures in three-dimensional fluid flows , 2001 .

[41]  Andrew J. Majda,et al.  Lessons in uncertainty quantification for turbulent dynamical systems , 2012 .

[42]  Vladimir I. Bogachev,et al.  Fokker-planck-kolmogorov Equations , 2015 .

[43]  G. Haller Finding finite-time invariant manifolds in two-dimensional velocity fields. , 2000, Chaos.

[44]  N. N. Chent︠s︡ov Statistical decision rules and optimal inference , 1982 .

[45]  V. Bogachev,et al.  Existence and uniqueness of solutions for Fokker–Planck equations on Hilbert spaces , 2009, 0907.1431.

[46]  Vladimir I. Bogachev,et al.  Distances between transition probabilities of diffusions and applications to nonlinear Fokker–Planck–Kolmogorov equations , 2016 .

[47]  Constance de Koning,et al.  Editors , 2003, Annals of Emergency Medicine.

[48]  Jerrold E. Marsden,et al.  Lagrangian coherent structures in n-dimensional systems , 2007 .

[49]  国田 寛 Stochastic flows and stochastic differential equations , 1990 .

[50]  Christian Léonard,et al.  Minimizers of energy functionals , 2001 .

[51]  O. Barndorff-Nielsen Parametric statistical models and likelihood , 1988 .

[52]  W. Rudin Real and complex analysis , 1968 .

[53]  Imre Csiszár,et al.  Axiomatic Characterizations of Information Measures , 2008, Entropy.

[54]  R. Tyrrell Rockafellar Conjugate Duality and Optimization , 1974 .

[55]  A. Monahan,et al.  Stochastic Averaging of Idealized Climate Models , 2011 .

[56]  M. Dellnitz,et al.  On the Approximation of Transport Phenomena – a Dynamical Systems Approach , 2009 .

[57]  Andrew J. Majda,et al.  Non-Gaussian Test Models for Prediction and State Estimation with Model Errors , 2013, Chinese Annals of Mathematics, Series B.

[58]  Annie Millet,et al.  Integration by Parts and Time Reversal for Diffusion Processes , 1989 .

[59]  T. S. Doan,et al.  Hopf bifurcation with additive noise , 2017, Nonlinearity.

[60]  Andrew J. Majda,et al.  Quantifying uncertainty for predictions with model error in non-Gaussian systems with intermittency , 2012 .

[61]  J. Marsden,et al.  Definition and properties of Lagrangian coherent structures from finite-time Lyapunov exponents in two-dimensional aperiodic flows , 2005 .

[62]  I. Csiszár A class of measures of informativity of observation channels , 1972 .

[63]  Djalil CHAFAÏ Entropies, convexity, and functional inequalities , 2002 .

[64]  Wolfram Just,et al.  Elimination of Fast Chaotic Degrees of Freedom: On the Accuracy of the Born Approximation , 2003 .

[65]  Mohammad Farazmand,et al.  Erratum and addendum to ''A variational theory of hyperbolic Lagrangian coherent structures'' (Physica D 240 (2011) 574-598) , 2012 .

[66]  P. Cattiaux,et al.  Large deviations and Nelson processes , 1995 .

[67]  J. Ottino The Kinematics of Mixing: Stretching, Chaos, and Transport , 1989 .

[68]  Stephen Wiggins,et al.  Chaotic transport in dynamical systems , 1991 .

[69]  Paul Dupuis,et al.  Robust Bounds on Risk-Sensitive Functionals via Rényi Divergence , 2013, SIAM/ASA J. Uncertain. Quantification.

[70]  D. W. Stroock,et al.  Multidimensional Diffusion Processes , 1979 .

[71]  R. Rockafellar Conjugate Duality and Optimization , 1987 .

[72]  Mtw,et al.  Stochastic flows and stochastic differential equations , 1990 .

[73]  Paul Malliavin,et al.  Stochastic Analysis , 1997, Nature.

[74]  Shun-ichi Amari,et al.  Information Geometry and Its Applications , 2016 .

[75]  Stephen Wiggins,et al.  A tutorial on dynamical systems concepts applied to Lagrangian transport in oceanic flows defined as finite time data sets: Theoretical and computational issues , 2006 .

[76]  Yannis Pantazis,et al.  Path-Space Information Bounds for Uncertainty Quantification and Sensitivity Analysis of Stochastic Dynamics , 2015, SIAM/ASA J. Uncertain. Quantification.

[77]  H. Chernoff A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[78]  P. Imkeller,et al.  Reduction of deterministic coupled atmosphere–ocean models to stochastic ocean models: a numerical case study of the Lorenz–Maas system , 2003 .

[79]  Michal Branicki,et al.  Information Theory for Climate Change and Prediction , 2015 .

[80]  C. Gardiner Stochastic Methods: A Handbook for the Natural and Social Sciences , 2009 .

[81]  Diffusion semigroups and diffusion processes corresponding to degenerate divergence form operators , 1997 .