Optimistic Bull or Pessimistic Bear: Adaptive Deep Reinforcement Learning for Stock Portfolio Allocation

Portfolio allocation is crucial for investment companies. However, getting the best strategy in a complex and dynamic stock market is challenging. In this paper, we propose a novel Adaptive Deep Deterministic Reinforcement Learning scheme (Adaptive DDPG) for the portfolio allocation task, which incorporates optimistic or pessimistic deep reinforcement learning that is reflected in the influence from prediction errors. Dow Jones 30 component stocks are selected as our trading stocks and their daily prices are used as the training and testing data. We train the Adaptive DDPG agent and obtain a trading strategy. The Adaptive DDPG's performance is compared with the vanilla DDPG, Dow Jones Industrial Average index and the traditional min-variance and mean-variance portfolio allocation strategies. Adaptive DDPG outperforms the baselines in terms of the investment return and the Sharpe ratio.

[1]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[2]  Ralph Neuneier,et al.  Enhancing Q-Learning for Optimal Asset Allocation , 1997, NIPS.

[3]  Xiao-Yang Liu,et al.  A Practical Machine Learning Approach for Dynamic Stock Recommendation , 2018, 2018 17th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/ 12th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE).

[4]  Yinchuan Li,et al.  Risk Management via Anomaly Circumvent: Mnemonic Deep Learning for Midterm Stock Prediction , 2019, ArXiv.

[5]  Myles E. Mangram,et al.  A Simplified Perspective of the Markowitz Portfolio Theory , 2013 .

[6]  M. Lebreton,et al.  Behavioural and neural characterization of optimistic reinforcement learning , 2017, Nature Human Behaviour.

[7]  Ling Liu,et al.  The effect of news and public mood on stock movements , 2014, Inf. Sci..

[8]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[9]  W. Sharpe.  Portfolio Theory and Capital Markets , 1970 .

[10]  Wenhang Bao,et al.  Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis , 2019, ArXiv.

[11]  Alexei A. Efros,et al.  Large-Scale Study of Curiosity-Driven Learning , 2018, ICLR.

[12]  M. Flannery,et al.  Macroeconomic Factors Do Influence Aggregate Stock Returns , 2002 .

[13]  Xiao-Yang Liu,et al.  Practical Deep Reinforcement Learning Approach for Stock Trading , 2018, ArXiv.

[14]  W. Sharpe.  The Sharpe Ratio , 1994 .

[15]  Tie-Yan Liu,et al.  Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Prediction , 2017, WSDM.

[16]  Pei-Chann Chang,et al.  Integrating a Piecewise Linear Representation Method and a Neural Network Model for Stock Trading Points Prediction , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).