Using Relative Entropy to Find Optimal Approximations: an Application to Simple Fluids

We develop a maximum relative entropy formalism to generate optimal approximations to probability distributions. The central results consist of (a) justifying the use of relative entropy as the uniquely natural criterion to select a preferred approximation from within a family of trial parameterized distributions, and (b) to obtain the optimal approximation by marginalizing over parameters using the method of maximum entropy and information geometry. As an illustration we apply our method to simple fluids. The “exact” canonical distribution is approximated by that of a fluid of hard spheres. The proposed method first determines the preferred value of the hard-sphere diameter, and then obtains an optimal hard-sphere approximation by a suitably weighed average over different hard-sphere diameters. This leads to a considerable improvement in accounting for the soft-core nature of the interatomic potential. As a numerical demonstration, the radial distribution function and the equation of state for a Lennard-Jones fluid (argon) are compared with results from molecular dynamics simulations.

[1]  J. Skilling.  Classic Maximum Entropy , 1989 .

[2]  N. N. Chent︠s︡ov.  Statistical decision rules and optimal inference , 1982 .

[3]  Miguel Robles,et al.  Thermodynamic perturbation theory and glass transition in simple fluids , 2006 .

[4]  Jean-Pierre Hansen,et al.  Phase Transitions of the Lennard-Jones System , 1969 .

[5]  E. Jaynes,et al.  E. T. Jaynes: Papers on Probability, Statistics and Statistical Physics , 1983 .

[6]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[7]  K A Dill,et al.  An improved thermodynamic perturbation theory for Mercedes-Benz water. , 2007, The Journal of chemical physics.

[8]  Ariel Caticha,et al.  Maximum Entropy Approach to the Theory of Simple Fluids , 2003, cond-mat/0310746.

[9]  R. Johnson,et al.  Properties of cross-entropy minimization , 1981, IEEE Trans. Inf. Theory.

[10]  D Chandler,et al.  Van der Waals Picture of Liquids, Solids, and Phase Transformations , 1983, Science.

[11]  J. Skilling.  The Axioms of Maximum Entropy , 1988 .

[12]  M. Wertheim,et al.  EXACT SOLUTION OF THE PERCUS-YEVICK INTEGRAL EQUATION FOR HARD SPHERES , 1963 .

[13]  Jerome K. Percus,et al.  Analysis of Classical Statistical Mechanics by Means of Collective Coordinates , 1958 .

[14]  J. Barker,et al.  What is "liquid"? Understanding the states of matter , 1976 .

[15]  C. Ray Smith,et al.  Maximum-entropy and Bayesian methods in science and engineering , 1988 .

[16]  Santos,et al.  Radial distribution function for hard spheres. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[17]  H. Callen.  Thermodynamics and an Introduction to Thermostatistics , 1988 .

[18]  I. Csiszár.  Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[19]  J.M.H. Levelt,et al.  The reduced equation of state, internal energy and entropy of argon and xenon , 1960 .

[20]  John Skilling,et al.  Maximum Entropy and Bayesian Methods , 1989 .

[21]  Ariel Caticha.  Maximum entropy, fluctuations and priors , 2001 .

[22]  S Amokrane,et al.  Validity of the perturbation theory for hard particle systems with very-short-range attraction. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[23]  Jerome Percus,et al.  Approximation Methods in Classical Statistical Mechanics , 1962 .

[24]  Richard J. Bearman,et al.  Numerical Solutions of the Percus—Yevick Equation for the Hard‐Sphere Potential , 1965 .

[25]  Ariel Caticha,et al.  Lectures on Probability, Entropy, and Statistical Physics , 2008, ArXiv.

[26]  Artur B Adib.  Algebraic perturbation theory for dense liquids with discrete potentials. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  Daan Frenkel,et al.  Comparison of simple perturbation-theory estimates for the liquid–solid and the liquid–vapor interfacial free energies of Lennard-Jones systems , 2007 .

[28]  Stanley I. Sandler,et al.  Equation of state for the Lennard–Jones fluid based on the perturbation theory , 2008 .

[29]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[30]  V. Kalikmanov,et al.  Statistical physics of fluids , 2001 .

[31]  Gerco Onderwater,et al.  AIP Conf. Proc. , 2009 .

[32]  P. Cummings,et al.  Fluid phase equilibria , 2005 .

[33]  Dominicus Kester,et al.  BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING , 2010 .

[34]  M. Tribus,et al.  Probability theory: the logic of science , 2003 .

[35]  Hua Lee,et al.  Maximum Entropy and Bayesian Methods. , 1996 .

[36]  U. von Toussaint,et al.  Bayesian inference and maximum entropy methods in science and engineering , 2004 .

[37]  G. A. Mansoori,et al.  Variational Approach to the Equilibrium Thermodynamic Properties of Simple Liquids. I , 1969 .

[38]  M. Wertheim,et al.  Analytic Solution of the Percus-Yevick Equation , 1964 .

[39]  J. Skilling.  Quantified Maximum Entropy , 1990 .

[40]  Yiping Tang,et al.  Improved expressions for the radial distribution function of hard spheres , 1995 .

[41]  Santos,et al.  Structure of hard-sphere metastable fluids. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[42]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[43]  R. Preuss,et al.  Maximum entropy and Bayesian data analysis: Entropic prior distributions. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[44]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[45]  Ariel Caticha,et al.  Change, time and information geometry , 2000 .

[46]  L. Verlet.  Computer "Experiments" on Classical Fluids. II. Equilibrium Correlation Functions , 1968 .

[47]  Ericka Stricklin-Parker,et al.  Ann , 2005 .

[48]  H. C. Andersen,et al.  Role of Repulsive Forces in Determining the Equilibrium Structure of Simple Liquids , 1971 .

[49]  A. Caticha.  Relative Entropy and Inductive Inference , 2003, physics/0311093.

[50]  Ariel Caticha,et al.  Maximum Entropy Approach to a Mean Field Theory for Fluids , 2003 .

[51]  E. Thiele,et al.  Equation of State for Hard Spheres , 1963 .