A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes*

Proper scoring rules provide convenient and highly efficient tools for incentive-compatible elicitations of subjective beliefs. As traditionally used, however, they are valid only under expected value maximization. This paper shows how they can be generalized to modern ("non-expected utility") theories of risk and ambiguity, yielding mutual benefits: users of scoring rules can benefit from the empirical realism of non-expected utility, and analysts of ambiguity attitudes can benefit from efficient measurements using proper scoring rules. An experiment demonstrates the feasibility of our generalization. Copyright 2009, Wiley-Blackwell.

[1]  George Wu,et al.  Composition rules in original and cumulative prospect theory , 2022 .

[2]  Peter P. Wakker,et al.  The Midweight Method to Measure Attitudes Toward Risk and Ambiguity , 2009, Manag. Sci..

[3]  Cma Training,et al.  Predictability of Weather and Climate , 2011 .

[4]  P. Wakker.  Prospect Theory: For Risk and Ambiguity , 2010 .

[5]  Christophe Gonzales,et al.  Multiattribute Utility Theory , 2010, Decision-making Process.

[6]  Thomas R. Palfrey,et al.  On Eliciting Beliefs in Strategic Games , 2009 .

[7]  Yoram Halevy,et al.  Uncertainty and compound lotteries: calibration , 2008, Economic Theory.

[8]  Jinkwon Lee,et al.  The effect of the background risk in a simple chance improving decision model , 2008 .

[9]  Wei Li,et al.  Changing One's Mind When the Facts Change: Incentives of Experts and the Design of Reporting Protocols , 2007 .

[10]  Edi Karni,et al.  A new approach to modeling decision-making under uncertainty , 2007 .

[11]  D. Johnstone.  The Value of a Probability Forecast from Portfolio Theory , 2007 .

[12]  Wojciech Olszewski,et al.  Preferences Over Sets of Lotteries -super-1 , 2007 .

[13]  David Johnstone,et al.  Economic Darwinism: Who has the Best Probabilities? , 2007 .

[14]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[15]  LI WEI,et al.  Changing One ’ s Mind when the Facts Change : Incentives of Experts and the Design of Reporting Protocols , 2007 .

[16]  D. Lindley.  Savage, Leonard J , 2006 .

[17]  Massimo Marinacci,et al.  AMBIGUITY AVERSION, ROBUSTNESS, AND THE VARIATIONAL , 2006 .

[18]  Robert F. Nau,et al.  Uncertainty Aversion with Second-Order Utilities and Probabilities , 2006, Manag. Sci..

[19]  Wojciech Olszewski,et al.  Preferences over Sets of Lotteries , 2006 .

[20]  C. Starmer,et al.  Preference Anomalies, Preference Elicitation and the Discovered Preference Hypothesis , 2005 .

[21]  Yoram Halevy.  Ellsberg Revisited: An Experimental Study , 2005 .

[22]  Mohammed Abdellaoui,et al.  Choice-Based Elicitation and Decomposition of Decision Weights for Gains and Losses Under Uncertainty , 2003, Manag. Sci..

[23]  A. Rustichini,et al.  Ambiguity Aversion, Robustness, and the Variational Representation of Preferences , 2006 .

[24]  Steven Gjerstad.  Risk Aversion, Beliefs, and Prediction Market Equilibrium , 2004 .

[25]  S. Chew,et al.  The Market for Sweepstakes , 2004 .

[26]  D. Prelec.  A Bayesian Truth Serum for Subjective Data , 2004, Science.

[27]  Itzhak Gilboa,et al.  Fact-Free Learning , 2004 .

[28]  Itzhak Gilboa,et al.  Uncertainty in economic theory : essays in honor of David Schmeidler's 65th birthday , 2004 .

[29]  J. Wolfers,et al.  Prediction Markets , 2003 .

[30]  Alan Greenspan,et al.  Risk and Uncertainty in Monetary Policy , 2004 .

[31]  Peter P Wakkar.  On the composition of risk preference and belief. , 2004, Psychological review.

[32]  Elyès Jouini,et al.  Consensus Consumer and Intertemporal Asset Pricing with Heterogeneous Beliefs , 2007 .

[33]  M. Marinacci,et al.  A Smooth Model of Decision Making Under Ambiguity , 2003 .

[34]  Gary Charness,et al.  When Optimal Choices Feel Wrong: A Laboratory Study of Bayesian Updating, Complexity, and Affect , 2004 .

[35]  Alvaro Sandroni,et al.  Calibration with Many Checking Rules , 2003, Math. Oper. Res..

[36]  P. Wakker,et al.  On the composition of risk preference and belief. , 2004, Psychological review.

[37]  J. Giles.  Scientific wagers: Wanna bet? , 2002, Nature.

[38]  Morten I. Lau,et al.  Estimating Individual Discount Rates in Denmark: A Field Experiment , 2002 .

[39]  A. Schotter,et al.  An Experimental Study of Belief Learning Using Elicited Beliefs , 2002 .

[40]  Jacob K. Goeree,et al.  Quantal Response Equilibrium and Overbidding in Private-Value Auctions , 2002, J. Econ. Theory.

[41]  Charles A. Holt,et al.  Risk Aversion and Incentive Effects , 2002 .

[42]  Steffen Huck,et al.  Do players correctly estimate what others do?: Evidence of conservatism in beliefs , 2002 .

[43]  Massimo Marinacci,et al.  APPLIED MATHEMATICS WORKING PAPER SERIES Risk, Ambiguity, and the Separation of Utility and Beliefs † , 2001 .

[44]  Mark J. Machina,et al.  Almost-objective uncertainty , 2001 .

[45]  G. Harcourt.  Keynes, John Maynard , 2001 .

[46]  Itzhak Gilboa,et al.  A theory of case-based decisions , 2001 .

[47]  J. L. Pinto,et al.  A Parameter-Free Elicitation of the Probability Weighting Function in Medical Decision Analysis , 2000 .

[48]  M. Abdellaoui.  Parameter-Free Elicitation of Utility and Probability Weighting Functions , 2000 .

[49]  C. Starmer.  Developments in Non-expected Utility Theory: The Hunt for a Descriptive Theory of Choice under Risk , 2000 .

[50]  R. Luce.  Utility of Gains and Losses: Measurement-Theoretical and Experimental Approaches , 2000 .

[51]  Sujoy Mukerji,et al.  Ambiguity Aversion and Incompleteness of Financial Markets , 2001 .

[52]  R. Selten,et al.  Money Does Not Induce Risk Neutral Behavior, but Binary Lotteries Do even Worse , 1999 .

[53]  T. Sargent,et al.  Robust Permanent Income and Pricing , 1999 .

[54]  V. Feltkamp,et al.  A Bayesian Approach to Uncertainty Aversion , 1999 .

[55]  David Kjellberg.  Measuring Expectations , 1999, Identification Problems in the Social Sciences.

[56]  Richard Gonzalez,et al.  On the Shape of the Probability Weighting Function , 1999, Cognitive Psychology.

[57]  D. Prelec.  The Probability Weighting Function , 1998 .

[58]  C. Plott,et al.  Exchange Economies and Loss Exposure: Experiments Exploring Prospect Theory and Competitive Equilibria in Market Environments , 1997 .

[59]  Ian J. Bateman,et al.  Does Part-Whole Bias Exist? An Experimental Investigation , 1997 .

[60]  Sujoy Mukerji,et al.  Understanding the nonadditive probability decision model , 1997 .

[61]  L. Perlemuter.  [From theory to practice]. , 1997, Soins. Psychiatrie.

[62]  P. Wakker,et al.  Eliciting von Neumann-Morgenstern Utilities When Probabilities Are Distorted or Unknown , 1996 .

[63]  R. Shiller,et al.  Why Did the Nikkei Crash? Expanding the Scope of Expectations Data Collection , 1996 .

[64]  A. Tversky,et al.  Support theory: A nonextensional representation of subjective probability. , 1994 .

[65]  Ab McClelland,et al.  The calibration of subjective probabilities: Theories and models 1980-1993 , 1994 .

[66]  Colin Camerer,et al.  Recent developments in modeling preferences: Uncertainty and ambiguity , 1992 .

[67]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[68]  D. Schmeidler,et al.  A More Robust Definition of Subjective Probability , 1992 .

[69]  S. Werlang,et al.  Uncertainty Aversion, Risk Aversion, and the Optimal Choice of Portfolio , 1992 .

[70]  G. Keren.  Calibration and probability judgements: Conceptual and methodological issues , 1991 .

[71]  R. Sugden.  Rational Choice: A Survey of Contributions from Economics and Philosophy , 1991 .

[72]  Faruk Gul.  A Theory of Disappointment Aversion , 1991 .

[73]  R. Duncan Luce,et al.  Rank- and sign-dependent linear utility models for binary gambles , 1991 .

[74]  R. McKelvey,et al.  Public and private information : an experimental study of information pooling , 1990 .

[75]  John Broome.  Bolker-Jeffrey Expected Utility Theory and Axiomatic Utilitarianism , 1990 .

[76]  R. Hogarth,et al.  Venture Theory: A Model of Decision Weights , 1990 .

[77]  Uzi Segal,et al.  First order versus second order risk aversion , 1990 .

[78]  R. Thaler,et al.  Gambling with the house money and trying to break even: the effects of prior outcomes on risky choice , 1990 .

[79]  Edi Karni,et al.  Revelations in auctions and the structure of preferences , 1989 .

[80]  David Schmeidleis.  SUBJECTIVE PROBABILITY AND EXPECTED UTILITY WITHOUT ADDITIVITY , 1989 .

[81]  I. Gilboa,et al.  Maxmin Expected Utility with Non-Unique Prior , 1989 .

[82]  John M. Miyamoto,et al.  Generic utility theory: measurement foundations and applications in multiattribute utility theory , 1988 .

[83]  William F. Wright,et al.  Empirical comparison of subjective probability elicitation methods , 1988 .

[84]  M. Machina.  Choice under Uncertainty: Problems Solved and Unsolved , 1987 .

[85]  Edi Karni,et al.  Preference reversal and the observability of preferences by experimental methods , 1987 .

[86]  Franklin Allen.  Notes---Discovering Personal Probabilities When Utility Functions are Unknown , 1987 .

[87]  I. Gilboa.  Expected utility with purely subjective non-additive probabilities , 1987 .

[88]  D J Spiegelhalter,et al.  Probabilistic prediction in patient management and clinical trials. , 1986, Statistics in medicine.

[89]  Charles A. Holt,et al.  Preference Reversals and the Independence Axiom , 1986 .

[90]  R. McKelvey,et al.  Common Knowledge, Consensus, and Aggregate Information , 1986 .

[91]  Howard Kunreuther,et al.  Ambiguity and Insurance Decisions , 1985 .

[92]  J. Quiggin.  A theory of anticipated utility , 1982 .

[93]  P. Schoemaker.  The Expected Utility Model: Its Variants, Purposes, Evidence and Limitations , 1982 .

[94]  L. Hurwicz.  Studies in Resource Allocation Processes: Optimality and informational efficiency in resource allocation processes , 1977 .

[95]  D. McFadden.  Quantal Choice Analysis: A Survey , 1976 .

[96]  A. H. Murphy,et al.  subjective probability forecasting experiments in meteorology: some preliminary results , 1974 .

[97]  A. K. Basu.  A Theory of Decision-Making , 1973, The Journal of Sociology & Social Welfare.

[98]  Carl-Axel S. Staël von Holstein,et al.  Probabilistic forecasting: An experiment related to the stock market , 1972 .

[99]  D. McFadden.  Conditional logit analysis of qualitative choice behavior , 1972 .

[100]  Gary Echternacht,et al.  The Use of Confidence Testing In Objective Tests , 1971 .

[101]  L. J. Savage.  Elicitation of Personal Probabilities and Expectations , 1971 .

[102]  A. H. Murphy,et al.  Nonlinear Utility and the Probability Score , 1970 .

[103]  D. Bernoulli.  Specimen theoriae novae de mensura sortis : translated into German and English , 1967 .

[104]  Thornton Page,et al.  The Scientist Speculates: An Anthology of Partly-baked Ideas , 1964 .

[105]  F. Sanders.  On Subjective Probability Forecasting , 1963 .

[106]  D. Ellsberg.  Decision, probability, and utility: Risk, ambiguity, and the Savage axioms , 1961 .

[107]  Johann Pfanzag.  A general theory of measurement applications to utility , 1959 .

[108]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .

[109]  L. J. Savage,et al.  The Foundations of Statistics , 1955 .

[110]  M. Allais.  Le comportement de l'homme rationnel devant le risque : critique des postulats et axiomes de l'ecole americaine , 1953 .

[111]  Frederick Mosteller,et al.  An Experimental Measurement of Utility , 1951, Journal of Political Economy.

[112]  Abraham Wald,et al.  Statistical Decision Functions , 1951 .

[113]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[114]  G. Shackle.  A Non-Additive Measure of Uncertainty , 1949 .

[115]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[116]  B. D. Finetti.  La prévision : ses lois logiques, ses sources subjectives , 1937 .

[117]  J. Keynes.  A Treatise on Probability. , 1923 .

[118]  F. Knight.  The economic nature of the firm: From Risk, Uncertainty, and Profit , 2009 .