Efficient algorithms of pathwise dynamic programming for decision optimization in mining operations

Complexity and uncertainty associated with commodity resource valuation and extraction requires stochastic control methods suitable for high dimensional states. Recent progress in duality and trajectory-wise techniques has introduced a variety of fresh ideas to this field with surprising results. This paper presents a concept which implements this promising development and illustrates it on a selection of traditional commodity extraction problems. We describe efficient algorithms for obtaining approximate solutions along with a diagnostic technique, which provides a quantitative measure for solution performance in terms of the distance between the approximate and the optimal control policy. All quantitative tools are efficiently implemented and are publicly available within a user friendly package in the statistical language R, which can help practitioners in a broad range of decision optimization problems.

[1]  Juri Hinz,et al.  Stochastic switching for partially observable dynamics and optimal asset allocation , 2017, Int. J. Control.

[2]  S. Ross A Simple Approach to the Valuation of Risky Streams , 1978 .

[3]  T. Björk Arbitrage Theory in Continuous Time , 2019 .

[4]  Erhan Bayraktar,et al.  On the One-Dimensional Optimal Switching Problem , 2007, Math. Oper. Res..

[5]  P. Glasserman,et al.  Pricing American-style securities using simulation , 1997 .

[6]  Francis A. Longstaff,et al.  Valuing American Options by Simulation: A Simple Least-Squares Approach , 2001 .

[7]  John N. Tsitsiklis,et al.  Optimal stopping of Markov processes: Hilbert space theory, approximation algorithms, and an application to pricing high-dimensional financial derivatives , 1999, IEEE Trans. Autom. Control..

[8]  Eduardo S. Schwartz The stochastic behavior of commodity prices: Implications for valuation and hedging , 1997 .

[9]  Juri Hinz,et al.  Using Convex Switching Techniques for Partially Observable Decision Processes , 2016, IEEE Transactions on Automatic Control.

[10]  J. Carriére Valuation of the early-exercise price for options using simulations and nonparametric regression , 1996 .

[11]  Andrianos E. Tsekrekos,et al.  Optimal Switching Decisions Under Stochastic Volatility with Fast Mean Reversion , 2016, Eur. J. Oper. Res..

[12]  Holger Paul Keeler,et al.  Bitcoin blockchain dynamics: The selfish-mine strategy in the presence of propagation delay , 2015, Perform. Evaluation.

[13]  E. Gobet,et al.  A regression-based Monte Carlo method to solve backward stochastic differential equations , 2005, math/0508491.

[14]  L. C. G. Rogers,et al.  Pathwise Stochastic Optimal Control , 2007, SIAM J. Control. Optim..

[15]  B. Øksendal,et al.  Applied Stochastic Control of Jump Diffusions , 2004, Universitext.

[16]  Jimmy E. Hilliard,et al.  Valuation of Commodity Futures and Options under Stochastic Convenience Yields, Interest Rates, and Jump Diffusions in the Spot , 1998, Journal of Financial and Quantitative Analysis.

[17]  Marco Antonio Guimarães Dias,et al.  Valuation of exploration and production assets: an overview of real options models , 2004 .

[18]  Milos Hauskrecht,et al.  Value-Function Approximations for Partially Observable Markov Decision Processes , 2000, J. Artif. Intell. Res..

[19]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[20]  Jeremy Yee,et al.  An Algorithmic Approach to Optimal Asset Liquidation Problems , 2017 .

[21]  Eduardo S. Schwartz,et al.  Short-Term Variations and Long-Term Dynamics in Commodity Prices , 2000 .

[22]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[23]  M. Shackleton,et al.  Evaluating Natural Resource Investments Under Different Model Dynamics: Managerial Insights , 2009 .

[24]  Daniel Egloff Monte Carlo algorithms for optimal stopping and statistical learning , 2004, math/0408276.

[25]  S. Bhattacharya PROJECT VALUATION WITH MEAN-REVERTING CASH FLOW STREAMS , 1978 .

[26]  Eduardo S. Schwartz,et al.  Optimal Exploration Investments Under Price and Geological-Technical Uncertainty: A Real Options Model , 2000 .

[27]  Emmanuel Gobet,et al.  Numerical simulation of BSDEs using empirical regression methods: theory and practice , 2005 .

[28]  Eduardo S. Schwartz,et al.  Evaluating Natural Resource Investments , 1985 .

[29]  Denis Belomestny,et al.  Regression Methods for Stochastic Control Problems and Their Convergence Analysis , 2009, SIAM J. Control. Optim..

[30]  Sudipto Sarkar,et al.  The effect of mean reversion on investment under uncertainty , 2003 .

[31]  Commodity Derivatives Valuation with Autoregressive and Moving Average Components in the Price Dynamics , 2010 .

[32]  A. Tsekrekos,et al.  The Effect of Mean Reversion on Entry and Exit Decisions Under Uncertainty , 2009 .

[33]  Juri Hinz,et al.  Optimal Stochastic Switching under Convexity Assumptions , 2014, SIAM J. Control. Optim..

[34]  Nikolai Dokuchaev,et al.  A First�?Order BSPDE for Swing Option Pricing , 2013 .

[35]  R. McDonald,et al.  Investment and the Valuation of Firms When There Is an Option to Shut Down , 1985 .

[36]  Mark Broadie,et al.  A Primal-Dual Simulation Algorithm for Pricing Multi-Dimensional American Options , 2001 .

[37]  Juri Hinz,et al.  Algorithms for Optimal Control of Stochastic Switching Systems , 2015 .

[38]  R. Carmona,et al.  OPTIMAL MULTIPLE STOPPING AND VALUATION OF SWING OPTIONS , 2008 .

[39]  Philip Protter,et al.  An analysis of a least squares regression method for American option pricing , 2002, Finance Stochastics.

[40]  Eduardo S. Schwartz,et al.  Stochastic Convenience Yield and the Pricing of Oil Contingent Claims , 1990 .

[41]  Paul Glasserman,et al.  Monte Carlo Methods in Financial Engineering , 2003 .

[42]  S. Jaimungal,et al.  Stepping Through Fourier Space , 2008 .

[43]  Philip Protter,et al.  An analysis of the Longstaff-Schwartz algorithm for American option pricing , 2001 .

[44]  Eduardo S. Schwartz,et al.  Investment Under Uncertainty. , 1994 .

[45]  James S. Dyer,et al.  Decision Analysis and Real Options: A Discrete Time Approach to Real Option Valuation , 2005, Ann. Oper. Res..

[46]  Trine Krogh Boomsma,et al.  Renewable energy investments under different support schemes: A real options approach , 2012, Eur. J. Oper. Res..

[47]  John N. Tsitsiklis,et al.  Regression methods for pricing complex American-style options , 2001, IEEE Trans. Neural Networks.

[48]  Peter W. Glynn,et al.  Kernel-Based Reinforcement Learning in Average-Cost Problems: An Application to Optimal Portfolio Choice , 2000, NIPS.

[49]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[50]  Michel Verschuere,et al.  Pricing electricity risk by interest rate methods , 2005 .

[51]  Nicola Secomandi,et al.  Comparison of Least Squares Monte Carlo Methods with Applications to Energy Real Options , 2017 .

[52]  Andrea Gamba,et al.  Real Options Valuation: a Monte Carlo Simulation Approach 1 , 2002 .

[53]  Ken Seng Tan,et al.  Applications of randomized low discrepancy sequences to the valuation of complex securities , 2000 .

[54]  Max Fehr,et al.  Storage Costs in Commodity Option Pricing , 2010, SIAM J. Financial Math..

[55]  R. Carmona,et al.  Valuation of energy storage: an optimal switching approach , 2010 .

[56]  Peng Sun,et al.  Information Relaxations and Duality in Stochastic Dynamic Programs , 2010, Oper. Res..

[57]  Warren B. Powell,et al.  “Approximate dynamic programming: Solving the curses of dimensionality” by Warren B. Powell , 2007, Wiley Series in Probability and Statistics.

[58]  Andrea Gamba,et al.  Real Options Valuation: A Monte Carlo Approach , 2003 .

[59]  M. Kohler,et al.  A dynamic look-ahead Monte Carlo algorithm for pricing Bermudan options , 2007, 0710.3640.

[60]  Juri Hinz,et al.  Optimal forward trading and battery control under renewable electricity generation , 2017, Journal of Banking & Finance.

[61]  L. Trigeorgis Real Options: Managerial Flexibility and Strategy in Resource Allocation , 1996 .

[63]  Jianqing Fan,et al.  Local polynomial modelling and its applications , 1994 .

[64]  Cyril Grunspan,et al.  Double spend races , 2017, International Journal of Theoretical and Applied Finance.

[65]  Lars Stentoft,et al.  Convergence of the Least Squares Monte Carlo Approach to American Option Valuation , 2004, Manag. Sci..

[66]  M. Fu,et al.  Pricing American Options: A Comparison of Monte Carlo Simulation Approaches ⁄ , 2001 .

[67]  L. Trigeorgis The Nature of Option Interactions and the Valuation of Investments with Multiple Real Options , 1993, Journal of Financial and Quantitative Analysis.

[68]  Amitabh Sinha,et al.  Integrated Optimization of Procurement, Processing, and Trade of Commodities , 2011, Oper. Res..

[69]  P. Collin‐Dufresne,et al.  Stochastic Convenience Yield Implied from Commodity Futures and Interest Rates , 2005 .

[70]  Dirk Ormoneit,et al.  Kernel-Based Reinforcement Learning , 2017, Encyclopedia of Machine Learning and Data Mining.

[71]  Sebastian Jaimungal,et al.  Real option pricing with mean-reverting investment and project value , 2010 .

[72]  U. Rieder,et al.  Markov Decision Processes with Applications to Finance , 2011 .

[73]  Huyen Pham,et al.  Continuous-time stochastic control and optimization with financial applications / Huyen Pham , 2009 .

[74]  N. Meinshausen,et al.  MONTE CARLO METHODS FOR THE VALUATION OF MULTIPLE‐EXERCISE OPTIONS , 2004 .

[75]  Model Uncertainty in Commodity Markets , 2015 .

[76]  H. Geman Commodities and Commodity Derivatives: Modelling and Pricing for Agriculturals, Metals and Energy , 2005 .

[77]  Sebastian Jaimungal,et al.  Levy Based Cross-Commodity Models and Derivative Valuation , 2011, SIAM J. Financial Math..