Expected utility maximization of optimal stopping problems

Abstract We consider the problem of optimal stopping of an independent and identically distributed sequence of random variables with observation costs and no recall for a decision maker, who maximizes expected utility. Though this problem does not fit into the framework of dynamic programming, we derive a recursive algorithm to find the optimal solution. Moreover, we show that the optimal stopping strategy has the reservation level property for all utility functions and we investigate how attitude to risk affects the solution. In the case of exponential utility we derive an easy recursion for the reservation levels and determine the range of its possible values. The results are illustrated by several examples.

[1]  R. Howard,et al.  Risk-Sensitive Markov Decision Processes , 1972 .

[2]  Paul Hudson Studies in the Economics of Search , 1979 .

[3]  K. Arrow Essays in the theory of risk-bearing , 1958 .

[4]  T. Stewart Optimal Selection from a Random Sequence with Learning of the Underlying Distribution , 1978 .

[5]  S. Lippman,et al.  THE ECONOMICS OF JOB SEARCH: A SURVEY* , 1976 .

[6]  P. Whittle Risk-Sensitive Optimal Control , 1990 .

[7]  J. Sonnemans,et al.  Strategies of search , 1998 .

[8]  M. Rothschild,et al.  Searching for the Lowest Price When the Distribution of Prices Is Unknown , 1974, Journal of Political Economy.

[9]  Edi Karni,et al.  Unbounded behaviorally consistent stopping rules , 1994 .

[10]  G. Simons Great Expectations: Theory of Optimal Stopping , 1973 .

[11]  Alfred Müller,et al.  Optimal selection from distributions with unknown parameters: Robustness of Bayesian models , 1996, Math. Methods Oper. Res..

[12]  Meir G. Kohn,et al.  The theory of search , 1974 .

[13]  J. Pratt RISK AVERSION IN THE SMALL AND IN THE LARGE11This research was supported by the National Science Foundation (grant NSF-G24035). Reproduction in whole or in part is permitted for any purpose of the United States Government. , 1964 .

[14]  Steven A. Lippman,et al.  The Economics of Job Search: A Survey: Part I , 1976 .

[15]  James C. Cox,et al.  Laboratory experiments with a finite-horizon job-search model , 1989 .

[16]  D. J. White Utility, probabilistic constraints, mean and variance of discounted rewards in Markov decision processes , 1987 .