Learning Financial Asset-Specific Trading Rules via Deep Reinforcement Learning

Generating asset-specific trading signals based on the financial conditions of the assets is one of the challenging problems in automated trading. Various asset trading rules are proposed experimentally based on different technical analysis techniques. However, these kind of trading strategies are profitable, extracting new asset-specific trading rules from vast historical data to increase total return and decrease the risk of portfolios is difficult for human experts. Recently, various deep reinforcement learning (DRL) methods are employed to learn the new trading rules for each asset. In this paper, a novel DRL model with various feature extraction modules is proposed. The effect of different input representations on the performance of the models is investigated and the performance of DRL-based models in different markets and asset situations is studied. The proposed model in this work outperformed the other state-of-the-art models in learning single asset-specific trading rules and obtained a total return of almost 262% in two years on a specific asset while the best state-of-the-art model get 78% on the same asset in the same time period.

[1]  Serdar Birogul,et al.  YOLO Object Recognition Algorithm and “Buy-Sell Decision” Model Over 2D Candlestick Charts , 2020, IEEE Access.

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  Kun Chang Lee,et al.  A causal knowledge-based expert system for planning an Internet-based stock trading system , 2012, Expert Syst. Appl..

[4]  Franklin Allen,et al.  Using genetic algorithms to find technical trading rules , 1999 .

[5]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .

[6]  Y. Ong,et al.  An empirical study of Genetic Programming generated trading rules in computerized stock trading service system , 2008, 2008 International Conference on Service Systems and Service Management.

[7]  Chao Luo,et al.  An Adaptive Financial Trading System Using Deep Reinforcement Learning With Candlestick Decomposing Features , 2020, IEEE Access.

[8]  Ying Shen,et al.  Time-driven feature-aware jointly deep reinforcement learning for financial signal representation and algorithmic trading , 2020, Expert Syst. Appl..

[9]  Niranjan Sapkota,et al.  Technical trading rules in the cryptocurrency market , 2020 .

[10]  Youyong Kong,et al.  Deep Direct Reinforcement Learning for Financial Signal Representation and Trading , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Zihao Zhang,et al.  Deep Reinforcement Learning for Trading , 2019, The Journal of Financial Data Science.

[12]  David Power,et al.  The profitability of moving average trading rules in South Asian stock markets , 2001 .

[13]  Kalok Chan,et al.  The profitability of technical trading rules in the Asian stock markets , 1995 .

[14]  Shingo Mabu,et al.  A genetic relation algorithm with guided mutation for the large-scale portfolio optimization , 2009, 2009 ICCAS-SICE.

[15]  T. Kasetkasem,et al.  Empowered PG in Forex Trading , 2020, 2020 17th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON).

[16]  J. Moody,et al.  Performance functions and reinforcement learning for trading systems and portfolios , 1998 .

[17]  Jianguo Liu,et al.  Deep Stock Representation Learning: From Candlestick Charts to Investment Decisions , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  Shangkun Deng,et al.  The profitability of Ichimoku Kinkohyo based trading rules in stock markets and FX markets , 2020 .

[19]  Werner Kristjanpoller,et al.  Generating trading rules on US Stock Market using strongly typed genetic programming , 2019, Soft Computing.

[20]  Jean-Yves Potvin,et al.  Generating trading rules on the stock markets with genetic programming , 2004, Comput. Oper. Res..

[21]  B. LeBaron,et al.  Simple Technical Trading Rules and the Stochastic Properties of Stock Returns , 1992 .

[22]  Damien Ernst,et al.  An Application of Deep Reinforcement Learning to Algorithmic Trading , 2020, Expert Syst. Appl..

[23]  P. Alam,et al.  R , 1823, The Herodotus Encyclopedia.

[24]  Xudong Lin,et al.  A novel CNN-DDPG based AI-trader: Performance and roles in business operations , 2019, Transportation Research Part E: Logistics and Transportation Review.

[25]  Anastasios Tefas,et al.  Deep Reinforcement Learning for Financial Trading Using Price Trailing , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Luigi Troiano,et al.  Adaptive stock trading strategies with deep reinforcement learning methods , 2020, Inf. Sci..

[27]  Werner Kristjanpoller,et al.  Strongly-typed genetic programming and fuzzy inference system: An embedded approach to model and generate trading rules , 2020, Appl. Soft Comput..

[28]  Xiao-Yang Liu,et al.  Practical Deep Reinforcement Learning Approach for Stock Trading , 2018, ArXiv.

[29]  Ismael Orquín-Serrano Predictive Power of Adaptive Candlestick Patterns in Forex Market. Eurusd Case , 2020 .

[30]  Ohm Sornil,et al.  Generating Trading Strategies Based on Candlestick Chart Pattern Characteristics , 2019 .

[31]  Shingo Mabu,et al.  GNP-Sarsa with subroutines for trading rules on stock markets , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[32]  Victor I. Chang,et al.  Application of deep reinforcement learning in stock trading strategies and stock forecasting , 2019, Computing.

[33]  P. Alam ‘A’ , 2021, Composites Engineering: An A–Z Guide.

[34]  P. Alam ‘G’ , 2021, Composites Engineering: An A–Z Guide.

[35]  W. Sharpe The Sharpe Ratio , 1994 .

[36]  Yen-Liang Chen,et al.  Mining associative classification rules with stock trading data - A GA-based method , 2010, Knowl. Based Syst..

[37]  Shingo Mabu,et al.  Genetic network programming with sarsa learning and its application to creating stock trading rules , 2007, 2007 IEEE Congress on Evolutionary Computation.

[38]  Shingo Mabu,et al.  A model of portfolio optimization using time adapting genetic network programming , 2010, Comput. Oper. Res..

[39]  P. Alam ‘T’ , 2021, Composites Engineering: An A–Z Guide.

[40]  Thomas G. Fischer,et al.  Reinforcement learning in financial markets - a survey , 2018 .

[41]  Rommy Pramudya,et al.  Efficiency of Technical Analysis for the Stock Trading , 2020 .

[42]  Luo Chao,et al.  An Adaptive Financial Trading System Using Deep Reinforcement Learning With Candlestick Decomposing Features , 2020, IEEE Access.