Using Reinforcement Learning in the Algorithmic Trading Problem

The development of reinforced learning methods has extended application to many areas including algorithmic trading. In this paper trading on the stock exchange is interpreted into a game with a Markov property consisting of states, actions, and rewards. A system for trading the fixed volume of a financial instrument is proposed and experimentally tested; this is based on the asynchronous advantage actor-critic method with the use of several neural network architectures. The application of recurrent layers in this approach is investigated. The experiments were performed on real anonymized data. The best architecture demonstrated a trading strategy for the RTS Index futures (MOEX:RTSI) with a profitability of 66% per annum accounting for commission. The project source code is available via the following link: this http URL

[1]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[2]  Yuxi Li,et al.  Deep Reinforcement Learning: An Overview , 2017, ArXiv.

[3]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[4]  R. Bellman.  A Markovian Decision Process , 1957 .

[5]  Stelios D. Bekiros,et al.  Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach , 2010 .

[6]  Andrzej Mostowski.  Review: A. A. Markov, The Theory of Algorithms , 1953 .

[7]  Yusen Zhan,et al.  Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer , 2016, IJCAI.

[8]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[9]  W. Sharpe.  The Sharpe Ratio , 1994 .

[10]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[11]  J. Moody,et al.  Performance functions and reinforcement learning for trading systems and portfolios , 1998 .

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[14]  Bolyai János Matematikai Társulat,et al.  Theory of algorithms , 1985 .

[15]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[16]  Youyong Kong,et al.  Deep Direct Reinforcement Learning for Financial Signal Representation and Trading , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Matthew Saffell,et al.  Learning to trade via direct reinforcement , 2001, IEEE Trans. Neural Networks.

[18]  Xin Du,et al.  Algorithm Trading using Q-Learning and Recurrent Reinforcement Learning , 2022 .

[19]  Youyong Kong,et al.  Sparse Coding-Inspired Optimal Trading System for HFT Industry , 2015, IEEE Transactions on Industrial Informatics.