Zipf's law, power laws and maximum entropy

Zipf's law, and power laws in general, have attracted and continue to attract considerable attention in a wide variety of disciplines - from astronomy to demographics to software structure to economics to linguistics to zoology, and even warfare. A recent model of random group formation [RGF] attempts a general explanation of such phenomena based on Jaynes' notion of maximum entropy applied to a particular choice of cost function. In the present article I argue that the cost function used in the RGF model is in fact unnecessarily complicated, and that power laws can be obtained in a much simpler way by applying maximum entropy ideas directly to the Shannon entropy subject only to a single constraint: that the average of the logarithm of the observable quantity is specified.

[1]  George Kingsley Zipf,et al.  The Psychobiology of Language , 2022 .

[2]  Ewan D. Tempero,et al.  Understanding the shape of Java software , 2006, OOPSLA '06.

[3]  Julian Havil.  Gamma: Exploring Euler's Constant , 2003 .

[4]  Sebastian Bernhardsson,et al.  Zipf's law unzipped , 2011, ArXiv.

[5]  A. Plastino,et al.  The workings of the maximum entropy principle in collective human behaviour , 2012, Journal of The Royal Society Interface.

[6]  Lewis F. Richardson,et al.  Statistics Of Deadly Quarrels , 1960 .

[7]  G. Hardy,et al.  The general theory of Dirichlet's series , 1916, The Mathematical Gazette.

[8]  Michael Batty,et al.  There is More than a Power Law in Zipf , 2012, Scientific Reports.

[9]  E. T. Jaynes,et al.  The Relation of Bayesian and Maximum Entropy Methods , 1988 .

[10]  R. P. J. Wagensberg.  The maximum entropy principle and the nature of fractals , 1998, cond-mat/9804257.

[11]  Margaret M. Gow,et al.  A Course in Pure Mathematics , 1974 .

[12]  Mark E. J. Newman,et al.  Power-Law Distributions in Empirical Data , 2007, SIAM Rev..

[13]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[14]  M. Newman.  Power laws, Pareto distributions and Zipf's law , 2005 .

[15]  R. Baierlein.  Probability Theory: The Logic of Science , 2004 .

[16]  M..  A Course of Pure Mathematics , 1909, Nature.

[17]  Didier Sornette,et al.  Theory of Zipf's Law and Beyond , 2009 .

[18]  Peter Nijkamp,et al.  Accessibility of Cities in the Digital Economy , 2004, cond-mat/0412004.

[19]  Richard A. Silverman,et al.  Elementary real and complex analysis , 1973 .

[20]  George Kingsley Zipf,et al.  Human behavior and the principle of least effort , 1949 .

[21]  C. Tsallis.  Possible generalization of Boltzmann-Gibbs statistics , 1988 .

[22]  A. Plastino,et al.  MaxEnt and dynamical information , 2012, 1201.0889.

[23]  Edwin T. Jaynes,et al.  Prior Probabilities , 1968, Encyclopedia of Machine Learning.

[24]  R. Schiffer.  Psychobiology of Language , 1986 .

[25]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[26]  Thermodynamics of urban population flows. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  A. Hernando,et al.  Variational principle underlying scale invariant social systems , 2012, The European Physical Journal B.