Some aspects of the history of Bayesian information processing

For many years, traditional Bayesian (TB) and information theoretic (IT) procedures for learning from data were viewed as distinctly different approaches. Derivations of the TB and IT learning models are reviewed and compared. Then the synthesis by Zellner in 1988 of the TB and IT learning models and generalizations of them are described along with descriptions of selected applications. Included are learning procedures that do not require use of likelihood functions and/or priors. Works by leading Bayesians and information theorists are cited and related to TB/IT issues.

[1]  Marcel Boyer,et al.  Bayesian Models in Economic Theory , 1984 .

[2]  Kenneth M. Hanson,et al.  Maximum Entropy and Bayesian Methods , 1996 .

[3]  Arnold Zellner Bayesian Method of Moments (BMOM) Analysis of Mean and Regression Models , 1996 .

[4]  Arnold Zellner,et al.  Bayesian Method of Moments (BMOM) Analysis of Parametric and Semiparametric Regression Models , 1998 .

[5]  K. Chaloner,et al.  A Bayesian approach to outlier detection and residual analysis , 1988 .

[6]  A. Zellner Remarks on a ‘critique’ of the Bayesian Method of Moments , 2001 .

[7]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[8]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[9]  George G. Judge,et al.  Econometric foundations , 2000 .

[10]  J. Ibrahim,et al.  On Optimality Properties of the Power Prior , 2003 .

[11]  Arnold Zellner,et al.  Bayesian Methods and Entropy in Economics and Econometrics , 1991 .

[12]  Harold T. Davis,et al.  The theory of econometrics , 1942 .

[13]  S. Geisser Remarks on the 'Bayesian' method of moments , 1999 .

[14]  Amos Golan,et al.  Comparison of maximum entropy and higher-order entropy estimators , 2002 .

[15]  E. Soofi Principal Information Theoretic Approaches , 2000 .

[16]  Ehsan S. Soofi,et al.  Information Theory and Bayesian Statistics , 1996 .

[17]  H. Jeffreys,et al.  Theory of probability , 1896 .

[18]  R. Silver Quantum Entropy Regularization , 1999 .

[19]  J. Lisman,et al.  Econometrics and Thermodynamics: A Remark on Davis' Theory of Budgets , 1949 .

[20]  K. Chaloner,et al.  Bayesian analysis in statistics and econometrics : essays in honor of Arnold Zellner , 1996 .

[21]  M. Degroot,et al.  Bayesian Analysis and Uncertainty in Economic Theory. , 1986 .

[22]  L. J. Savage The Foundations of Statistical Inference. , 1963 .

[23]  L. Wasserman,et al.  The Selection of Prior Distributions by Formal Rules , 1996 .

[24]  J. Eatwell,et al.  The New Palgrave: Econometrics. , 1991 .

[25]  Teddy Seidenfeld Entropy and Uncertainty , 1986, Philosophy of Science.

[26]  Inferring the Nutrient Content of Food With Prior Information , 1999 .

[27]  Arnold Zellner,et al.  Bayesian Analysis in Econometrics and Statistics: The Zellner View and Papers , 1997 .

[28]  D. Schmeidler,et al.  A More Robust Definition of Subjective Probability , 1992 .

[29]  William E. Strawderman,et al.  A Bayesian growth and yield model for slash pine plantations , 1996 .

[30]  G. Barnard The Theory of Information , 1951 .

[31]  P. Jizba,et al.  The world according to R enyi: thermodynamics of multifractal systems , 2002, cond-mat/0207707.

[32]  A. Zellner Optimal Information Processing and Bayes's Theorem , 1988 .

[33]  BAYESIAN MODELING OF ECONOMIES AND DATA REQUIREMENTS , 2001, Macroeconomic Dynamics.

[34]  Son-Nan Chen,et al.  Estimation risk and optimal portfolios , 1985 .

[35]  Arnold Zellner,et al.  Further Results on Bayesian Method of Moments Analysis of the Multiple Regression Model , 2001 .

[36]  J. Bernardo Expected Information as Expected Utility , 1979 .

[37]  D. A. Sprott,et al.  Foundations of Statistical Inference. , 1972 .

[38]  Arnold Zellner,et al.  Bayesian Analysis of Regression Error Terms , 1975 .

[39]  Arnold Zellner,et al.  The finite sample properties of simultaneous equations' estimates and estimators Bayesian and non-Bayesian approaches , 1998 .

[40]  James D. Malley,et al.  Quantum Statistical Inference , 1993 .

[41]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[42]  Arnold Zellner,et al.  Prediction and Decision Problems in Regression Models from the Bayesian Point of View , 1965 .

[43]  A. Zellner Generalizing the standard product rule of probability theory and Bayes's Theorem , 2007 .

[44]  Arnold Zellner,et al.  Information processing and Bayesian analysis , 2002 .