Financial Optimization Problems

The theory of Markov Decision Processes which has been presented in Chapter 2 will now be applied to some selected dynamic optimization problems in finance. The basic underlying model is the financial market of Chapter 3. We will always assume that investors are small and cannot influence the asset price process. We begin in the first two sections with the classical problem of maximizing the expected utility of terminal wealth. In Section 4.1 we consider the general one-period model. It will turn out that the existence of an optimal portfolio strategy is equivalent to the absence of arbitrage in this market. Moreover, the one-period problem is the key building block for the multiperiod problems which are investigated in Section 4.2 and which can be solved with the theory of Markov Decision Processes. In this section we will also present some results for special utility functions and the relation to continuous-time models is highlighted. In Section 4.3 consumption and investment problems are treated and solved explicitly for special utility functions. The next section generalizes these models to include regime switching. Here a Markov chain is used to model the changing economic conditions which give rise to a changing return distribution. Under some simplifying assumptions this problem is solved and the influence of the environment is discussed. Section 4.5 deals with models with proportional transaction cost. For homogeneous utility functions it will turn out that the action space is separated into sell-, buy- and no-transaction regions which are defined by cones. The next section considers dynamic mean-variance problems. In contrast to utility functions the idea is now to measure the risk by the portfolio variance and to search among all portfolios which yield at least a certain expected return, the one with smallest portfolio variance. The challenge is here to reduce this problem to a Markov Decision Problem first. Essentially the task boils down to solving a linear-quadratic problem. In Section 4.7 the variance is replaced by the risk measure ‘Average-Value-at-Risk’. In order to obtain an explicit solution in this mean-risk model, only the binomial model is considered here and the relation to the mean-variance problem is discussed. Section 4.8 deals with index-tracking problems and Section 4.9 investigates the problem of indifference pricing in incomplete markets. Finally, the last but one section explains the relation to continuous-time models and introduces briefly the approximating Markov chain approach. The last section contains some remarks and references.