Optimal Decision Stimuli for Risky Choice Experiments: An Adaptive Approach

Collecting data to discriminate between models of risky choice requires careful selection of decision stimuli. Models of decision making aim to predict decisions across a wide range of possible stimuli, but practical limitations force experimenters to select only a handful of them for actual testing. Some stimuli are more diagnostic between models than others, so the choice of stimuli is critical. This paper provides the theoretical background and a methodological framework for adaptive selection of optimal stimuli for discriminating among models of risky choice. The approach, called Adaptive Design Optimization (ADO), adapts the stimulus in each experimental trial based on the results of the preceding trials. We demonstrate the validity of the approach with simulation studies aiming to discriminate Expected Utility, Weighted Expected Utility, Original Prospect Theory, and Cumulative Prospect Theory models.

[1]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[2]  John R. Hauser,et al.  Active Machine Learning for Consideration Heuristics , 2011, Mark. Sci..

[3]  M. Leek.  Adaptive procedures in psychophysical research , 2001, Perception & psychophysics.

[4]  Jay I. Myung,et al.  Optimal experimental design for model discrimination. , 2009, Psychological review.

[5]  Christian P. Robert,et al.  Bayesian-Optimal Design via Interacting Particle Systems , 2006 .

[6]  Anita Rao,et al.  Adaptive Self-Explication of Multiattribute Preferences , 2011 .

[7]  M. Allais.  Le comportement de l'homme rationnel devant le risque : critique des postulats et axiomes de l'ecole americaine , 1953 .

[8]  Gutti Jogesh Babu,et al.  Statistical Challenges of Astronomy , 2003 .

[9]  Michael L. Littman,et al.  A tutorial on partially observable Markov decision processes , 2009 .

[10]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[11]  Mark J. Garratt,et al.  Efficient Experimental Design with Marketing Research Applications , 1994 .

[12]  Zhong-Lin Lu,et al.  Bayesian adaptive estimation of the contrast sensitivity function: the quick CSF method. , 2010, Journal of vision.

[13]  H. Stott.  Cumulative prospect theory's functional menagerie , 2006 .

[14]  Ken Binmore,et al.  Experimental Economics: Science or What? , 2007 .

[15]  M. Machina.  "Expected Utility" Analysis without the Independence Axiom , 1982 .

[16]  Robert Sugden,et al.  A Microeconometric Test of Alternative Stochastic Theories of Risky Choice , 2002 .

[17]  Daniel R. Cavagnaro,et al.  Discriminating among probability weighting functions using adaptive design optimization , 2013, Journal of risk and uncertainty.

[18]  Tuomas J. Lukka,et al.  Bayesian adaptive estimation: The next dimension , 2006 .

[19]  Heinz Holling,et al.  Advances in optimum experimental design for conjoint analysis and discrete choice models , 2002 .

[20]  J. Kruschke.  Bayesian approaches to associative learning: From passive to active learning , 2008, Learning & behavior.

[21]  A. Brix.  Bayesian Data Analysis, 2nd edn , 2005 .

[22]  M. Birnbaum,et al.  New Paradoxes of Risky Decision Making , 2022 .

[23]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[24]  M. Birnbaum,et al.  Testing for intransitivity of preferences predicted by a lexicographic semi-order , 2007 .

[25]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[26]  Michael D. Lee,et al.  Optimal experimental design for a class of bandit problems , 2010 .

[27]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[28]  Xinwei Deng,et al.  Active Learning Through Sequential Design, With Applications to Detection of Money Laundering , 2009 .

[29]  H. Fennema,et al.  Original and cumulative prospect theory: a discussion of empirical differences , 1997 .

[30]  W. Näther.  Optimum experimental designs , 1994 .

[31]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[32]  P. Blavatskyy.  Stochastic expected utility theory , 2005 .

[33]  Michael H. Birnbaum,et al.  A Comparison of Five Models that Predict Violations of First-Order Stochastic Dominance in Risky Decision Making , 2005 .

[34]  Oded Netzer,et al.  Adaptive Self-Explication of Multiattribute Preferences , 2011 .

[35]  Anthony C. Atkinson,et al.  Optimum Experimental Designs , 1992 .

[36]  A. Atkinson,et al.  Optimal design : Experiments for discriminating between several models , 1975 .

[37]  T. Loredo.  Bayesian Adaptive Exploration , 2004, astro-ph/0409386.

[38]  Joshua B. Tenenbaum,et al.  Inferring causal networks from observations and interventions , 2003, Cogn. Sci..

[39]  Mark A. Pitt,et al.  Adaptive Design Optimization: A Mutual Information-Based Approach to Model Discrimination in Cognitive Science , 2010, Neural Computation.

[40]  John D. Hey,et al.  Why We Should Not Be Silent About Noise , 2005, Experiments in Economics.

[41]  J. Hey,et al.  INVESTIGATING GENERALIZATIONS OF EXPECTED UTILITY THEORY USING EXPERIMENTAL DATA , 1994, Experiments in Economics.

[42]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[43]  Peter Goos,et al.  Models and Optimal Designs for Conjoint Choice Experiments Including a No-Choice Option , 2008 .

[44]  P. Müller,et al.  Optimal Bayesian Design by Inhomogeneous Markov Chain Simulation , 2004 .

[45]  Luis A. Lesmes,et al.  Bayesian adaptive estimation of threshold versus contrast external noise functions: The quick TvC method , 2006, Vision Research.

[46]  Tapabrata Maiti,et al.  Bayesian Data Analysis (2nd ed.) (Book) , 2004 .

[47]  Chris D. Orme,et al.  Investigating Generalisations of Expected Utility Theory Using Experimental Data , 1994 .

[48]  George Wu,et al.  Testing Prospect Theories Using Probability Tradeoff Consistency , 2005 .

[49]  Rémi Bardenet,et al.  Monte Carlo Methods , 2013, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[50]  Jay I. Myung,et al.  Model discrimination through adaptive experimentation , 2011, Psychonomic bulletin & review.

[51]  Robert Sugden,et al.  Incorporating a stochastic element into decision theories , 1995 .

[52]  David W Harless,et al.  The predictive utility of generalized expected utility theories , 1994 .

[53]  J. Marschak.  Rational Behavior, Uncertain Prospects, and Measurable Utility (1950) , 1950 .

[54]  Colin Camerer.  An experimental test of several generalized utility theories , 1989 .

[55]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[56]  Gary H. McClelland,et al.  Optimal design in psychological research. , 1997 .

[57]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[58]  I. J. Myung,et al.  The Importance of Complexity in Model Selection. , 2000, Journal of mathematical psychology.

[59]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[60]  C. S. Hong,et al.  Empirical Tests of Weighted Utility Theory , 1986 .

[61]  C. Starmer.  Developments in Non-expected Utility Theory: The Hunt for a Descriptive Theory of Choice under Risk , 2000 .

[62]  Peter Müller,et al.  Bayesian Optimal Design for Phase II Screening Trials , 2008, Biometrics.

[63]  S. Chew.  A Generalization of the Quasilinear Mean with Applications to the Measurement of Income Inequality and Decision Theory Resolving the Allais Paradox , 1983 .

[64]  J. Timmer,et al.  Systems biology: experimental design , 2009, The FEBS journal.

[65]  M. Abdellaoui.  Parameter-Free Elicitation of Utility and Probability Weighting Functions , 2000 .

[66]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[67]  L. Haines,et al.  Bayesian Optimal Designs for Phase I Clinical Trials , 2003, Biometrics.

[68]  Jacob Marschak,et al.  Stochastic models of choice behavior , 2007 .

[69]  P. Fishburn.  SSB Utility theory: an economic perspective , 1984 .

[70]  A. Atkinson,et al.  The design of experiments for discriminating between two rival models , 1975 .

[71]  Richard Gonzalez,et al.  Common Consequence Conditions in Decision Making under Risk , 1998 .

[72]  Robert J. Butera,et al.  Sequential Optimal Design of Neurophysiology Experiments , 2009, Neural Computation.

[73]  P. Laycock,et al.  Optimum Experimental Designs , 1995 .

[74]  Dennis J. Aigner,et al.  A brief introduction to the methodology of optimal experimental design , 1979 .

[75]  D. Ellsberg.  Decision, probability, and utility: Risk, ambiguity, and the Savage axioms , 1961 .