Portfolio trading system of digital currencies: A deep reinforcement learning with multidimensional attention gating mechanism

Abstract As a hot topic in the financial engineering, the portfolio optimization aims to increase investors’ wealth. In this paper, a portfolio management system based on deep-reinforcement learning is proposed. In contrast to inflexible traditional methods, the proposed system achieves a better trading strategy through Reinforcement learning. The reward signal of Reinforcement learning is updated by action weights from Deep learning networks. Low price, high price and close price constitute the inputs, but the importance of these three features is quite different. Traditional methods and the classical CNN can’t deal with these three features separately, but in our method, a designed depth convolution is proposed to deal with these three features separately. In a virtual currency market, the price rise only occurs in a flash. Traditional methods and CNN networks can’t accurately judge the critical time. In order to solve this problem, a three-dimensional attention gating network is proposed and it gives higher weights on rising moments and assets. Under different market conditions, the proposed system achieves more substantial returns and greatly improves the Sharpe ratios. The short-term risk index of the proposed system is lower than those of the traditional algorithms. Simulation results show that the traditional algorithms (including Best, CRP, PAMR, CWMR and CNN) are unable to perform as well as our approach.

[1]  Xu Zhang,et al.  Non-intrusive load disaggregation based on deep dilated residual network , 2019, Electric Power Systems Research.

[2]  Peter W. Glynn,et al.  Kernel-Based Reinforcement Learning in Average-Cost Problems: An Application to Optimal Portfolio Choice , 2000, NIPS.

[3]  Liguo Weng,et al.  Cloud/snow recognition for multispectral satellite imagery based on a multidimensional deep residual network , 2018, International Journal of Remote Sensing.

[4]  Zhengyao Jiang,et al.  Cryptocurrency portfolio management with deep reinforcement learning , 2016, 2017 Intelligent Systems Conference (IntelliSys).

[5]  Simon Caton,et al.  Predicting the Price of Bitcoin Using Machine Learning , 2018, 2018 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP).

[6]  Steve Y. Yang,et al.  An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown , 2017, Expert Syst. Appl..

[7]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[8]  Steve Y. Yang,et al.  A constrained portfolio trading system using particle swarm algorithm and recurrent reinforcement learning , 2019, Expert Syst. Appl..

[9]  Min Xia,et al.  Weighted Densely Connected Convolutional Networks for Reinforcement Learning , 2020, Int. J. Pattern Recognit. Artif. Intell..

[10]  John L. Kelly,et al.  A new interpretation of information rate , 1956, IRE Trans. Inf. Theory.

[11]  Can Berk Kalayci,et al.  A comprehensive review of deterministic models and applications for mean-variance portfolio optimization , 2019, Expert Syst. Appl..

[12]  Jaewook Lee,et al.  An Empirical Study on Modeling and Prediction of Bitcoin Prices With Bayesian Neural Networks Based on Blockchain Information , 2018, IEEE Access.

[13]  Steven C. H. Hoi,et al.  PAMR: Passive aggressive mean reversion strategy for portfolio selection , 2012, Machine Learning.

[14]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[15]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Tom Schaul,et al.  Rainbow: Combining Improvements in Deep Reinforcement Learning , 2017, AAAI.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Allan Borodin,et al.  Can We Learn to Beat the Best Stock , 2003, NIPS.

[19]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[20]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Bin Li,et al.  Confidence Weighted Mean Reversion Strategy for Online Portfolio Selection , 2011, TKDD.

[22]  Andrea Baronchelli,et al.  Machine Learning the Cryptocurrency Market , 2018, Complex..

[23]  Steven C. H. Hoi,et al.  Online portfolio selection: A survey , 2012, CSUR.