A maximum likelihood algorithm for the estimation and renormalization of exponential densities

We present an algorithm based on maximum likelihood for the estimation and renormalization (marginalization) of exponential densities. The moment-matching problem resulting from the maximization of the likelihood is solved as an optimization problem using the Levenberg-Marquardt algorithm. In the case of renormalization, the moments needed to set up the moment-matching problem are evaluated using Swendsen's renormalization method. We focus on the renormalization version of the algorithm, where we demonstrate its use by computing the critical temperature of the two-dimensional Ising model. Possible applications of the algorithm are discussed.

[1]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[2]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[3]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[4]  Alexandre J. Chorin,et al.  Optimal prediction with memory , 2002 .

[5]  Julian Besag,et al.  Markov Chain Monte Carlo for Statistical Inference , 2002 .

[6]  Mario Stefanelli,et al.  Contribution to the discussion of the paper by Steffen L. Lauritzen and David Spiegelhalter: "Local Computations with Probabilities on Graphical Structures and their Application to Expert Systems" , 1988 .

[7]  Michael I. Jordan.  Graphical Models , 2003 .

[8]  C. Geyer,et al.  Constrained Monte Carlo Maximum Likelihood for Dependent Data , 1992 .

[9]  L. Brown.  Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[10]  Alexandre J. Chorin.  Conditional Expectations and Renormalization , 2003, Multiscale Model. Simul..

[11]  A. Fisher,et al.  The Theory of Critical Phenomena: An Introduction to the Renormalization Group , 1992 .

[12]  Open problems in Monte Carlo renormalization group: Application to critical phenomena (invited) , 1986 .

[13]  G. Jona-Lasinio.  Renormalization group and probability theory , 2000, cond-mat/0009219.

[14]  Lawrence D. Brown.  Fundamentals of Statistical Exponential Families , 1987 .

[15]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .