A Representation of the Relative Entropy with Respect to a Diffusion Process in Terms of Its Infinitesimal Generator

In this paper we derive an integral (with respect to time) representation of the relative entropy (or Kullback-Leibler Divergence) R(µ||P), where µ and P are measures on C([0, T ]; R d). The underlying measure P is a weak solution to a Martingale Problem with continuous coefficients. Our representation is in the form of an integral with respect to its infinitesimal generator. This representation is of use in statistical inference (particularly involving medical imaging). Since R(µ||P) governs the exponential rate of convergence of the empirical measure (according to Sanov's Theorem), this representation is also of use in the numerical and analytical investigation of finite-size effects in systems of interacting diffusions.

[1]  G. B. Arous,et al.  Large deviations for Langevin spin glass dynamics , 1995 .

[2]  H. Zessin,et al.  Ergodic theorems for spatial processes , 1979 .

[3]  Peter Grandits,et al.  On the minimal entropy martingale measure , 2002 .

[4]  Mimimal Relative Entropy Martingale Measure of Birth and Death Process , 2000 .

[5]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[6]  Tryphon T. Georgiou,et al.  Kullback-Leibler approximation of spectral density functions , 2003, IEEE Trans. Inf. Theory.

[7]  P. Dupuis,et al.  Large deviation properties of weakly interacting processes via weak convergence methods , 2010, 1009.6030.

[8]  Minh N. Do,et al.  Wavelet-based texture retrieval using generalized Gaussian density and Kullback-Leibler distance , 2002, IEEE Trans. Image Process..

[9]  R. Lassalle.  Invertibility of adapted perturbations of the identity on abstract Wiener space , 2012 .

[10]  Olivier Faugeras,et al.  A large deviation principle for networks of rate neurons with correlated synaptic weights , 2013, BMC Neuroscience.

[11]  Prashant G. Mehta,et al.  The Kullback-Leibler rate metric for comparing dynamical systems , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[12]  M. Samuelides,et al.  Large deviations and mean-field theory for asymmetric random recurrent neural networks , 2002 .

[13]  Markus Fischer,et al.  On the form of the large deviation rate function for the empirical measures of weakly interacting systems , 2012, 1208.0472.

[14]  Minimal Relative Entropy Martingale Measures of Geometric Levy Processes and Option Pricing Models in Incomplete Markets , 2000 .

[15]  A. R. Plastino,et al.  MINIMUM KULLBACK ENTROPY APPROACH TO THE FOKKER-PLANCK EQUATION , 1997 .

[16]  M. Yor,et al.  Continuous martingales and Brownian motion , 1990 .

[17]  Entropy, Invertibility and Variational Calculus of the Adapted Shifts on Wiener space , 2009 .

[18]  M. Frittelli.  The Minimal Entropy Martingale Measure and the Valuation Problem in Incomplete Markets , 2000 .

[19]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[20]  A. S. Ustunel.  Entropy, Invertibility and Variational Calculus of the Adapted Shifts on Wiener space , 2009, 0903.3891.

[21]  C. Villani,et al.  On the trend to global equilibrium in spatially inhomogeneous entropy‐dissipating systems: The linear Fokker‐Planck equation , 2001 .

[22]  Xiongzhi Chen.  Brownian Motion and Stochastic Calculus , 2008 .

[23]  H. Akaike.  Likelihood of a model and information criteria , 1981 .

[24]  S. Varadhan,et al.  Asymptotic evaluation of certain Markov process expectations for large time , 1975 .

[25]  H. Bozdogan,et al.  Akaike's Information Criterion and Recent Developments in Information Complexity. , 2000, Journal of mathematical psychology.

[26]  Olivier Faugeras,et al.  Mean Field description of and propagation of chaos in recurrent multipopulation networks of Hodgkin-Huxley and Fitzhugh-Nagumo neurons , 2011, 1110.4294.

[27]  Olivier Faugeras,et al.  Large Deviations of an Ergodic Neural Network with Hebbian Learning , 2014 .

[28]  J. Gärtner,et al.  Large deviations from the mckean-vlasov limit for weakly interacting diffusions , 1987 .