Stochastic switching for partially observable dynamics and optimal asset allocation

ABSTRACT In industrial applications, optimal control problems frequently appear in the context of decision-making under incomplete information. In such framework, decisions must be adapted dynamically to account for possible regime changes of the underlying dynamics. Using stochastic filtering theory, Markovian evolution can be modelled in terms of latent variables, which naturally leads to high-dimensional state space, making practical solutions to these control problems notoriously challenging. In our approach, we utilise a specific structure of this problem class to present a solution in terms of simple, reliable, and fast algorithms. The algorithms presented in this paper have already been implemented in an R package.

[1]  U. Rieder,et al.  Markov Decision Processes with Applications to Finance , 2011 .

[2]  G. Monahan.  State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[3]  George E. Monahan,et al.  A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 2007 .

[4]  L. C. G. Rogers,et al.  Pathwise Stochastic Optimal Control , 2007, SIAM J. Control. Optim..

[5]  Michael Z. Zgurovsky,et al.  Partially Observable Total-Cost Markov Decision Processes with Weakly Continuous Transition Probabilities , 2014, Math. Oper. Res..

[6]  Jeremy Yee,et al.  Algorithmic Solutions for Optimal Switching Problems , 2016, 2016 Second International Symposium on Stochastic Models in Reliability Engineering, Life Science and Operations Management (SMRLO).

[7]  Martin B. Haugh,et al.  Pricing American Options: A Duality Approach , 2001, Oper. Res..

[8]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[9]  O. Hernondex-lerma,et al.  Adaptive Markov Control Processes , 1989 .

[10]  Eugene A. Feinberg,et al.  Handbook of Markov Decision Processes , 2002 .

[11]  L. Rogers.  Monte Carlo valuation of American options , 2002 .

[12]  Juri Hinz,et al.  Solving Control Problems with Linear State Dynamics - A Practical User Guide , 2016, 2016 Second International Symposium on Stochastic Models in Reliability Engineering, Life Science and Operations Management (SMRLO).

[13]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[14]  Robert J. Elliott,et al.  PORTFOLIO OPTIMIZATION, HIDDEN MARKOV MODELS AND TECHNICAL ANALYSIS OF P&F CHARTS , 2002 .

[15]  James E. Smith,et al.  Dynamic Portfolio Optimization with Transaction Costs: Heuristics and Dual Bounds , 2011, Manag. Sci..

[16]  Milos Hauskrecht,et al.  Value-Function Approximations for Partially Observable Markov Decision Processes , 2000, J. Artif. Intell. Res..

[17]  Juri Hinz,et al.  Optimal Stochastic Switching under Convexity Assumptions , 2014, SIAM J. Control. Optim..

[18]  D. Rhenius.  Incomplete Information in Markovian Decision Models , 1974 .

[19]  Mark Broadie,et al.  A Primal-Dual Simulation Algorithm for Pricing Multi-Dimensional American Options , 2001 .

[20]  Juri Hinz,et al.  Algorithms for Optimal Control of Stochastic Switching Systems , 2015 .

[21]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[22]  Chun Wang,et al.  Dynamic Portfolio Execution and Information Relaxations , 2014, SIAM J. Financial Math..

[23]  Enlu Zhou,et al.  Optimal Stopping of Partially Observable Markov Processes: A Filtering-Based Duality Approach , 2013, IEEE Transactions on Automatic Control.

[24]  S. Rajagopalan,et al.  A Multiperiod Model of Inventory Competition , 2009 .

[25]  Juri Hinz,et al.  Using Convex Switching Techniques for Partially Observable Decision Processes , 2016, IEEE Transactions on Automatic Control.

[26]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[27]  Peng Sun,et al.  Information Relaxations and Duality in Stochastic Dynamic Programs , 2010, Oper. Res..

[28]  Andrew E. B. Lim,et al.  Linear-quadratic control and information relaxations , 2012, Oper. Res. Lett..

[29]  A. Yushkevich.  Reduction of a Controlled Markov Model with Incomplete Data to a Problem with Complete Information in the Case of Borel State and Control Space , 1976 .

[30]  Mark Broadie,et al.  ANNIVERSARY ARTICLE: Option Pricing: Valuation Models and Applications , 2004, Manag. Sci..

[31]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .