Statistical thermodynamics of natural images.

The scale invariance of natural images suggests an analogy to the statistical mechanics of physical systems at a critical point. Here we examine the distribution of pixels in small image patches and show how to construct the corresponding thermodynamics. We find evidence for criticality in a diverging specific heat, which corresponds to large fluctuations in how "surprising" we find individual images, and in the quantitative form of the entropy vs energy. We identify special image configurations as local energy minima and show that average patches within each basin are interpretable as lines and edges in all orientations.

[1]  Tobias J. Hagge,et al.  Physics , 1929, Nature.

[2]  L. Onsager.  Crystal statistics. I. A two-dimensional model with an order-disorder transition , 1944 .

[3]  Claude E. Shannon,et al.  The mathematical theory of communication , 1950 .

[4]  L. Kadanoff.  Scaling laws for Ising models near T(c) , 1966 .

[5]  M. Fisher,et al.  Bounded and Inhomogeneous Ising Models. I. Specific-Heat Anomaly of a Finite Lattice , 1969 .

[6]  F. Crick.  Thinking about the brain. , 1979, Scientific American.

[7]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Jensen,et al.  Time ordering and the thermodynamics of strange sets: Theory and experimental tests. , 1986, Physical review letters.

[10]  Mitchell J. Feigenbaum.  Some characterizations of strange sets , 1987 .

[11]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[12]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[13]  Nicolas Sourlas,et al.  Spin-glass models as error-correcting codes , 1989, Nature.

[14]  C. Beck,et al.  Thermodynamics of chaotic systems , 1993 .

[15]  C. Beck,et al.  Thermodynamics of chaotic systems : an introduction , 1993 .

[16]  William Bialek,et al.  Statistics of Natural Images: Scaling in the Woods , 1993, NIPS.

[17]  D. Ruderman.  The statistics of natural images , 1994 .

[18]  J. Cardy.  Scaling and Renormalization in Statistical Physics , 1996 .

[19]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[20]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[21]  N. C. Singh,et al.  Estimating spatio-temporal receptive fields of auditory and visual neurons from their responses to natural stimuli , 2001 .

[22]  William Bialek,et al.  Thinking About The Brain , 2002, physics/0205030.

[23]  Ben Willmore,et al.  The Receptive-Field Organization of Simple Cells in Primary Visual Cortex of Ferrets under Natural Scene Stimulation , 2003, The Journal of Neuroscience.

[24]  E. R. Davies,et al.  Machine vision - theory, algorithms, practicalities , 2004 .

[25]  M. Newman.  Power laws, Pareto distributions and Zipf's law , 2005 .

[26]  宁北芳,et al.  疟原虫var基因转换速率变化导致抗原变异[英]／Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A , 2005 .

[27]  M. E. J. Newman,et al.  Power laws, Pareto distributions and Zipf's law , 2005 .

[28]  Eero P. Simoncelli,et al.  Spatiotemporal Elements of Macaque V1 Receptive Fields , 2005, Neuron.

[29]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[30]  D. Field,et al.  Estimates of the information content and dimensionality of natural scenes from proximity distributions. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[31]  R. Rosenfeld.  Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[32]  W. Bialek,et al.  Are Biological Systems Poised at Criticality? , 2010, 1012.2242.

[33]  Joseph J Atick,et al.  Could information theory provide an ecological theory of sensory processing? , 2011, Network.

[34]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[35]  G. Zipf.  Selected Studies of the Principle of Relative Frequency in Language , 2014 .