Eliciting Beliefs: Theory and Experiments

Subjective beliefs play a role in many economic decisions. There is a large theoretical literature on the elicitation of beliefs, and an equally large empirical literature. However, there is a gulf between the two. The theoretical literature proposes a range of procedures that can be used to recover beliefs, but stresses the need to make strong auxiliary assumptions or “calibrating adjustments” to elicited reports in order to recover the latent belief. With some notable exceptions, the empirical literature seems intent on either making those strong assumptions or ignoring the need for calibration. We make three contributions to bridge this gulf. First, we offer a general theoretical framework in which the belief elicitation task can be viewed as an exchange of state-dependent commodities between two traders. Second, we provide a specific elicitation procedure which has clear counterparts in field betting environments, and that is directly motivated by our theoretical framework. Finally, we illustrate how one can jointly estimate risk attitudes and subjective beliefs using structural maximum likelihood methods. This allows the observer to make inferences about the latent subjective belief, calibrating for virtually any well-specified model of choice under uncertainty. We demonstrate our procedures with an experiment in which we elicit subjective probabilities over three future events and one fact. † Department of Economics and Centre for Economic and Business Research, Copenhagen Business School, Copenhagen, Denmark (Andersen); Department of Economics, University of Canterbury, Christchurch, New Zealand (Fountain); and Department of Economics, College of Business Administration, University of Central Florida, USA (Harrison and Rutström). Harrison is also affiliated with the Durham Business School, Durham University, UK. E-mail contacts: sa.eco@cbs.dk, john.fountain@canterbury.ac.nz, gharrison@research.bus.ucf.edu and erutstrom@bus.ucf.edu. We thank the U.S. National Science Foundation for research support under grants NSF/IIS 9817518, NSF/HSD 0527675 and NSF/SES 0616746, and the Danish Social Science Research Council for research support under project #24-02-0124. We are grateful to Peter Wakker for many helpful discussions. All instructions and statistical code are available at the ExLab Digital Library at http://exlab.bus.ucf.edu.

[1]  Steffen Andersen,et al.  Lost in State Space: Are Preferences Stable? , 2008 .

[2]  J. Bernardo Expected Information as Expected Utility , 1979 .

[3]  C. Manski Interpreting the Predictions of Prediction Markets , 2004 .

[4]  Morten I. Lau,et al.  Elicitation using multiple price list formats , 2006 .

[5]  M. Zeelenberg,et al.  Eliciting decision weights by adapting de Finetti’s betting-odds method to prospect theory , 2007 .

[6]  Henry E. Kyburg Objective Probabilities , 1987, IJCAI.

[7]  D. Fudenberg,et al.  A Dual Self Model of Impulse Control , 2004, The American economic review.

[8]  Uzi Segal,et al.  The Ellsberg Paradox and Risk Aversion: An Anticipated Utility Approach , 1987 .

[9]  Edi Karni Elicitation of Subjective Probabilities when Preferences are State‐Dependent , 1999 .

[10]  Richard A. Epstein,et al.  The Theory of Gambling and Statistical Logic , 1977 .

[11]  G. Harrison Hypothetical Bias Over Uncertain Outcomes , 2005 .

[12]  Martin Dufwenberg,et al.  Measuring Beliefs in an Experimental Lost Wallet Game , 2000, Games Econ. Behav..

[13]  G. Harrison,et al.  Expected utility theory and prospect theory: one wedding and a decent funeral , 2009 .

[14]  W. Hurley,et al.  A Note on the Hayek Hypothesis and the Favorite-Longshot Bias in Parimutuel Betting , 1995 .

[15]  Evan L. Porteus,et al.  Temporal Resolution of Uncertainty and Dynamic Choice Theory , 1978 .

[16]  A. H. Murphy,et al.  Nonlinear Utility and the Probability Score , 1970 .

[17]  Robin Hanson,et al.  Combinatorial Information Market Design , 2003, Inf. Syst. Frontiers.

[18]  Martin Dufwenberg,et al.  Author's Personal Copy Games and Economic Behavior the Framing of Games and the Psychology of Play , 2022 .

[19]  David M. Grether,et al.  Testing Bayes Rule and the Representativeness Heuristic: Some Experimental Evidence , 1992 .

[20]  F. Lad Operational Subjective Statistical Methods: A Mathematical, Philosophical, and Historical Introduction , 1996 .

[21]  Todd L. Cherry,et al.  Hardnose the Dictator , 2002 .

[22]  Morten I. Lau,et al.  Eliciting Risk and Time Preferences , 2008 .

[23]  R. L. Winkler,et al.  Separating probability elicitation from utilities , 1988 .

[24]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[25]  Glenn W. Harrison,et al.  Eliciting risk and time preferences using field experiments: Some methodological issues , 2006 .

[26]  David Lindley,et al.  de Finetti, Bruno , 2005 .

[27]  E. Elisabet Rutström,et al.  Entitlements and fairness:: an experimental study of distributive preferences , 2000 .

[28]  J. Fountain Measuring Risk Relatively , 2002 .

[29]  Richard A. Epstein,et al.  The Theory of Gambling and Statistical Logic , 1968 .

[30]  Zvi Safra,et al.  The impossibility of experimental elicitation of subjective probabilities , 1995 .

[31]  L. J. Savage Elicitation of Personal Probabilities and Expectations , 1971 .

[32]  Rachel Croson,et al.  Thinking like a game theorist: factors affecting the frequency of equilibrium play , 2000 .

[33]  C. Noussair,et al.  Traders' Expectations in Asset Markets: Experimental Evidence , 2007 .

[34]  R. McKelvey,et al.  Public and private information : an experimental study of information pooling , 1990 .

[35]  D. Schmeidler Subjective Probability and Expected Utility without Additivity , 1989 .

[36]  N. Wilcox,et al.  Stated Beliefs Versus Empirical Beliefs : A Methodological Inquiry and Experimental Test , 2006 .

[37]  Bruno de Finetti,et al.  Logical foundations and measurement of subjective probability , 1970 .

[38]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .

[39]  G. Harrison Theory and Misbehavior of First-Price Auctions , 1989 .

[40]  J. Sonnemans,et al.  Value Orientations, Expectations and Voluntary Contributions in Public Goods. , 1996 .

[41]  D. Lindley Savage, Leonard J , 2006 .

[42]  J. Marschak,et al.  Actual versus consistent decision behavior. , 1964, Behavioral science.

[43]  Thomas S. Gruca,et al.  The Effect of Electronic Markets on Forecasts of New Product Success , 2003, Inf. Syst. Frontiers.

[44]  Peter P. Wakker,et al.  An index of loss aversion , 2005, J. Econ. Theory.

[45]  Belle Liang,et al.  Warts and All , 2004 .

[46]  E. Elisabet Rutström,et al.  Decision Making Costs and Problem Solving Performance , 2001 .

[47]  G. Harrison,et al.  Risk Aversion in the Laboratory , 2008 .

[48]  B. D. Finetti,et al.  Foresight: Its Logical Laws, Its Subjective Sources , 1992 .

[49]  Stephen E. Fienberg,et al.  Degroot, Morris H. , 2006 .

[50]  R. Hanson Correction to McKelvey and Page, "Public and Private Information: An Experimental Study of Information Pooling" , 1996 .

[51]  Barry Sopher,et al.  Social Learning and Coordination Conventions in Intergenerational Games: An Experimental Study , 2003, Journal of Political Economy.

[52]  Steven Gjerstad Risk Aversion, Beliefs, and Prediction Market Equilibrium , 2004 .

[53]  P. Wakker,et al.  Eliciting von Neumann-Morgenstern Utilities When Probabilities Are Distorted or Unknown , 1996 .

[54]  J. Sonnemans,et al.  A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes* , 2009 .

[55]  J. Quiggin A theory of anticipated utility , 1982 .

[56]  Robert Forsythe,et al.  Anatomy of an Experimental Political Stock Market , 1992 .

[57]  Martin Dufwenberg,et al.  Promises and Partnership , 2006 .

[58]  Charles A. Holt Scoring-Rule Procedures for Eliciting Subjective Probability and Utility Functions , 1999 .

[59]  Leslie E. Papke,et al.  Econometric Methods for Fractional Response Variables with an Application to 401(K) Plan Participation Rates , 1993 .

[60]  Allan H. Murphy,et al.  A Geometrical Framework for the Ranked Probability Score , 1975 .

[61]  J. Hey,et al.  INVESTIGATING GENERALIZATIONS OF EXPECTED UTILITY THEORY USING EXPERIMENTAL DATA , 1994, Experiments in Economics.

[62]  Eric J. Johnson,et al.  Risk Aversion and Incentive Effects: Comment , 2004 .

[63]  Winston C. Yang,et al.  Parimutuel betting markets as information aggregation devices: experimental results , 2003 .

[64]  J. Jaffray,et al.  Elicitation of Subjective Probabilities when the Initial Endowment is Unobservable , 1999 .

[65]  Jacob Marschak,et al.  Stochastic models of choice behavior , 2007 .

[66]  D. Schmeidler,et al.  A More Robust Definition of Subjective Probability , 1992 .

[67]  S. Katti Operational Subjective Statistical Methods , 1998 .

[68]  Charles A. Holt,et al.  Risk Aversion and Incentive Effects , 2002 .

[69]  A. Schotter,et al.  An Experimental Study of Belief Learning Using Elicited Beliefs , 2002 .

[70]  Miguel A. Costa-Gomes,et al.  Stated Beliefs and Play in Normal-Form Games , 2007 .

[71]  Allan H. Murphy,et al.  The Family of Quadratic Scoring Rules , 1978 .

[72]  M. Abdellaoui Parameter-Free Elicitation of Utility and Probability Weighting Functions , 2000 .

[73]  Mohammed Abdellaoui,et al.  Loss Aversion Under Prospect Theory: A Parameter-Free Measurement , 2007, Manag. Sci..

[74]  B. D. Finetti La prévision : ses lois logiques, ses sources subjectives , 1937 .

[75]  G. Oehlert A note on the delta method , 1992 .

[76]  J. Shogren,et al.  An Experimental Comparison of Induced and Elicited Beliefs , 2005 .

[77]  Daniel Read,et al.  Can intertemporal choice experiments elicit time preferences for consumption? , 2007 .

[78]  Robert L. Winkler,et al.  Scoring Rules, Generalized Entropy, and Utility Maximization , 2008, Oper. Res..

[79]  Glenn W. Harrison,et al.  Valuation using multiple price list formats , 2005 .