Cryptocurrency portfolio management with deep reinforcement learning

Portfolio management is the decision-making process of allocating an amount of fund into different financial investment products. Cryptocurrencies are electronic and decentralized alternatives to government-issued money, with Bitcoin as the best-known example of a cryptocurrency. This paper presents a model-less convolutional neural network with historic prices of a set of financial assets as its input, outputting portfolio weights of the set. The network is trained with 0.7 years' price data from a cryptocurrency exchange. The training is done in a reinforcement manner, maximizing the accumulative return, which is regarded as the reward function of the network. Back test trading experiments with trading period of 30 minutes is conducted in the same market, achieving 10-fold returns in 1.8 month's periods. Some recently published portfolio selection strategies are also used to perform the same back tests, whose results are compared with the neural network. The network is not limited to cryptocurrency, but can be applied to any other financial markets.

[1]  Steven C. H. Hoi,et al.  PAMR: Passive aggressive mean reversion strategy for portfolio selection , 2012, Machine Learning.

[2]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[3]  O. Blanchard,et al.  Speculative bubbles, crashes and rational expectations , 1979 .

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  W. Sharpe.  CAPITAL ASSET PRICES: A THEORY OF MARKET EQUILIBRIUM UNDER CONDITIONS OF RISK* , 1964 .

[6]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[7]  A. Stuart,et al.  Portfolio Selection: Efficient Diversification of Investments , 1959 .

[8]  Alexander J. Smola,et al.  Efficient mini-batch training for stochastic optimization , 2014, KDD.

[9]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[11]  W. Sharpe,et al.  Capital Asset Prices: A Theory of Market Equilibrium under Conditions of Risk , 2007 .

[12]  Steven C. H. Hoi,et al.  Online portfolio selection: A survey , 2012, CSUR.

[13]  Reuben Grinberg.  Bitcoin: An Innovative Alternative Digital Currency , 2011 .

[14]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[15]  Jeremy Clark,et al.  SoK: Research Perspectives and Challenges for Bitcoin and Cryptocurrencies , 2015, 2015 IEEE Symposium on Security and Privacy.

[16]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[17]  Iddo Bentov,et al.  Proof of Activity: Extending Bitcoin's Proof of Work via Proof of Stake [Extended Abstract]y , 2014, PERV.

[18]  Amir F. Atiya,et al.  The maximum drawdown of the Brownian motion , 2003, 2003 IEEE International Conference on Computational Intelligence for Financial Engineering, 2003. Proceedings..

[19]  C H HoiSteven,et al.  Online portfolio selection , 2014 .

[20]  Arindam Banerjee,et al.  Meta optimization and its application to portfolio selection , 2011, KDD.

[21]  Evan Duffield,et al.  Darkcoin : Peer to Peer Crypto Currency with Anonymous Blockchain Transactions and an Improved Proof of Work System , .

[22]  Thomas M. Cover,et al.  Universal data compression and portfolio selection , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[23]  M A H Dempster,et al.  An automated FX trading system using adaptive reinforcement learning , 2006, Expert Syst. Appl..

[24]  Jan Hendrik Witte,et al.  Deep Learning for Finance: Deep Portfolios , 2016 .

[25]  Vladimir Vovk,et al.  Universal portfolio selection , 1998, COLT' 98.

[26]  Matthew Saffell,et al.  Learning to trade via direct reinforcement , 2001, IEEE Trans. Neural Networks.

[27]  Seyed Taghi Akhavan Niaki,et al.  Forecasting S&P 500 index using artificial neural networks and design of experiments , 2013 .

[28]  G. Lugosi,et al.  NONPARAMETRIC KERNEL‐BASED SEQUENTIAL INVESTMENT STRATEGIES , 2006 .

[29]  John L. Kelly,et al.  A new interpretation of information rate , 1956, IRE Trans. Inf. Theory.

[30]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[31]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[32]  R. Haugen.  Modern investment theory , 1986 .

[33]  Kamran Usmani.  An Investigation into the Use of Reinforcement Learning Techniques within the Algorithmic Trading Domain , 2015 .