Deep Hedging: Hedging Derivatives Under Generic Market Frictions Using Reinforcement Learning

This article discusses a new application of reinforcement learning: to the problem of hedging a portfolio of “over-the-counter” derivatives under under market frictions such as trading costs and liquidity constraints. It is an extended version of our recent work https://www.ssrn.com/abstract=3120710, here using notation more common in the machine learning literature. The objective is to maximize a non-linear risk-adjusted return function by trading in liquid hedging instruments such as equities or listed options. The approach presented here is the first efficient and model-independent algorithm which can be used for such problems at scale.

[1]  Lizhong Wu,et al.  Optimization of trading systems and portfolios , 1997, Proceedings of the IEEE/IAFE 1997 Computational Intelligence for Financial Engineering (CIFEr).

[2]  H. Föllmer,et al.  Stochastic Finance: An Introduction in Discrete Time , 2002 .

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  David W. Lu,et al.  Agent Inspired Trading Using Recurrent Reinforcement Learning and LSTM Neural Networks , 2017, 1707.07338.

[5]  H. Soner,et al.  There is no nontrivial hedging portfolio for option pricing with transaction costs , 1995 .

[6]  Igor Halperin.  QLBS: Q-Learner in the Black-Scholes (-Merton) Worlds , 2017, ArXiv.

[7]  Xin Du,et al.  Algorithm Trading using Q-Learning and Recurrent Reinforcement Learning , 2022 .

[8]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[9]  Francis A. Longstaff,et al.  Valuing American Options by Simulation: A Simple Least-Squares Approach , 2001 .

[10]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[11]  F. Black,et al.  The Pricing of Options and Corporate Liabilities , 1973, Journal of Political Economy.

[12]  S. Heston.  A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options , 1993 .

[13]  Peter Bank,et al.  Hedging with temporary price impact , 2015, 1510.03223.

[14]  S. Peng,et al.  Backward Stochastic Differential Equations in Finance , 1997 .

[15]  Thomas G. Dietterich.  Adaptive computation and machine learning , 1998 .

[16]  Zhengyao Jiang,et al.  A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem , 2017, ArXiv.

[17]  L. Rogers,et al.  THE COST OF ILLIQUIDITY AND ITS EFFECTS ON HEDGING , 2010 .