Sequential Choice Under Ambiguity: Intuitive Solutions to the Armed-Bandit Problem

The process by which individuals learn from feedback when making recurrent choices among ambiguous alternatives is explored. We describe an experiment in which subjects solve a variant of the classic armed-bandit problem of dynamic decision theory, set in the context of airline choice. Subjects are asked to make repeated choices between two hypothetical airlines, one having an on-time departure probability which is known a priori, and the other has an ambiguous probability whose true value can only be discovered by making sample trips on the airline. Subjects attempt to make choices in such a way as to maximize the total number of one-time departures over a fixed planning horizon. We examine the extent to which actual choice patterns over time are consistent with those which would be made by a decision maker acting as an optimal Bernoulli sampler. The data offer support for a number of expected-and some unexpected-departures from optimality, including a tendency to underexperiment with promising options and overexperiment with unpromising options, and a tendency to increasingly switch between airlines as the average base rate of departures decreases. Implications of the work for the descriptive validity of normative dynamic decision models is explored, as well as for the generalizability of previous findings about choice under ambiguity to dynamic settings.

[1]  F RESTLE,et al.  The selection of strategies in cue learning. , 1962, Psychological review.

[2]  Harley Bornbach,et al.  An introduction to mathematical learning theory , 1967 .

[3]  M. Degroot.  Optimal Statistical Decisions , 1970 .

[4]  D. Berry.  A Bernoulli Two-armed Bandit , 1972 .

[5]  A. Tversky,et al.  On the psychology of prediction , 1973 .

[6]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[7]  R. Thaler.  Toward a positive theory of consumer choice , 1980 .

[8]  Preferences Scientific.  Kahneman, Daniel, and Amos Tversky. , 1982 .

[9]  Robert A. Miller.  Job Matching and Occupational Choice , 1984, Journal of Political Economy.

[10]  Michael Kolonko,et al.  The Sequential Design of Bernoulli Experiments Including Switching Costs , 1985, Oper. Res..

[11]  John W. Payne,et al.  Effort and Accuracy in Choice , 1985 .

[12]  R. Hogarth,et al.  Ambiguity and Uncertainty in Probabilistic Inference. , 1985 .

[13]  Irwin P. Levin,et al.  More than Meets the Eye: The Effect of Missing Information on Purchase Evaluations , 1985 .

[14]  A. Tversky,et al.  The hot hand in basketball: On the misperception of random sequences , 1985, Cognitive Psychology.

[15]  G. Northcraft,et al.  Opportunity costs and the framing of resource allocation decisions , 1986 .

[16]  Rakesh K. Sarin,et al.  Modeling Ambiguity in Decisions Under Uncertainty , 1988 .

[17]  K. Wolpin,et al.  The Specification and Estimation of Dynamic Stochastic Discrete Choice Models: A Survey , 1989 .

[18]  Shawn P. Curley,et al.  An empirical evaluation of descriptive models of ambiguity reactions in choice situations , 1989 .

[19]  A. Tversky,et al.  Preference and belief: Ambiguity and competence in choice under uncertainty , 1991 .

[20]  Colin Camerer,et al.  Recent developments in modeling preferences: Uncertainty and ambiguity , 1992 .

[21]  Eric J. Johnson,et al.  The adaptive decision maker , 1993 .

[22]  Colin Camerer,et al.  Cognition and framing in sequential bargaining for gains and losses , 1993 .