A novel CNN-DDPG based AI-trader: Performance and roles in business operations

Abstract Artificial Intelligence (AI) is well-developed as a part of human life. In both financial markets and business operations, AI is getting more and more important. In this paper, we build a novel “Reinforcement Learning” (RL) framework based AI-trader. We adopt an actor-critic RL algorithm called “Deep Deterministic Policy Gradient” (DDPG) to find the optimal policy. Our proposed DDPG has two different convolutional neutral networks (CNNs) based function approximators. The proposed AI-trader’s performance is shown to outperform other methods with the use of real stock-index future data. We further discuss the generalization and implications of the proposed method for business operations.

[1]  Maxime C. Cohen.  Big Data and Service Operations , 2018 .

[2]  A. Murat Ozbayoglu,et al.  Algorithmic financial trading with deep convolutional neural networks: Time series to image conversion approach , 2018, Appl. Soft Comput..

[3]  Ruonan Rao,et al.  A Multi-objective Deep Reinforcement Learning Approach for Stock Index Future’s Intraday Trading , 2017, 2017 10th International Symposium on Computational Intelligence and Design (ISCID).

[4]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[5]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[6]  Tsan-Ming Choi,et al.  Big Data Analytics in Operations Management , 2018 .

[7]  Bin Yu,et al.  Flight delay prediction for commercial air transport: A deep learning approach , 2019, Transportation Research Part E: Logistics and Transportation Review.

[8]  John N. Tsitsiklis,et al.  Actor-Critic Algorithms , 1999, NIPS.

[9]  Yong Yu,et al.  Sales forecasting using extreme learning machine with applications in fashion retailing , 2008, Decis. Support Syst..

[10]  Dimitris A. Tsouknidis,et al.  A survey of shipping finance research: Setting the future research agenda , 2018, Transportation Research Part E: Logistics and Transportation Review.

[11]  Lukas Menkhoff.  The use of technical analysis by fund managers: International evidence , 2010 .

[12]  T. Choi,et al.  The mean-variance approach for global supply chain risk analysis with air logistics in the blockchain technology era , 2019, Transportation Research Part E: Logistics and Transportation Review.

[13]  David de la Fuente,et al.  Neural networks in financial trading , 2019, Annals of Operations Research.

[14]  Matthew Saffell,et al.  Learning to trade via direct reinforcement , 2001, IEEE Trans. Neural Networks.

[15]  A. Murat Ozbayoglu,et al.  A deep learning based stock trading model with 2-D CNN trend detection , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[16]  Niraj Kumar,et al.  Understanding big data analytics capabilities in supply chain management: Unravelling the issues, challenges and implications for practice , 2017, Transportation Research Part E: Logistics and Transportation Review.

[17]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[18]  Guofu Zhou,et al.  Forecasting the Equity Risk Premium: The Role of Technical Indicators , 2011, Manag. Sci..

[19]  Michael A. H. Dempster,et al.  Computational learning techniques for intraday FX trading using popular technical indicators , 2001, IEEE Trans. Neural Networks.

[20]  Babak Nadjar Araabi,et al.  Online Forecasting of Synchronous Time Series Based on Evolving Linear Models , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[21]  Nishikant Mishra,et al.  Social media data analytics to improve supply chain management in food industries , 2017, Transportation Research Part E: Logistics and Transportation Review.

[22]  Ebrahimi Atani Reza,et al.  Stock market forecasting using artificial neural networks , 2014 .

[23]  M A H Dempster,et al.  An automated FX trading system using adaptive reinforcement learning , 2006, Expert Syst. Appl..

[24]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[25]  Marco Corazza,et al.  Testing different Reinforcement Learning con?gurations for ?nancial trading: Introduction and applications , 2018 .

[26]  T. Choi.  Incorporating social media observations and bounded rationality into fashion quick response supply chains in the big data era , 2016, Transportation Research Part E: Logistics and Transportation Review.

[27]  Pierre Baldi,et al.  From Reinforcement Learning to Deep Reinforcement Learning: An Overview , 2017, Braverman Readings in Machine Learning.

[28]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[29]  Lucas P. Veelenturf,et al.  The strategic role of logistics in the industry 4.0 era , 2019, Transportation Research Part E: Logistics and Transportation Review.

[30]  Raymond Y. K. Lau,et al.  Parallel Aspect‐Oriented Sentiment Analysis for Sales Forecasting with Big Data , 2018 .

[31]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[32]  Tsan-Ming Choi,et al.  Supply chain risk analysis with mean-variance models: a technical review , 2016, Ann. Oper. Res..

[33]  Sergey Levine,et al.  Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Jonghun Park,et al.  A Multiagent Approach to $Q$-Learning for Daily Stock Trading , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[35]  Tilman Matteis,et al.  A machine learning approach for the operationalization of latent classes in a discrete shipment size choice model , 2019, Transportation Research Part E: Logistics and Transportation Review.

[36]  Hing Kai Chan,et al.  Forecasting the demand of the aviation industry using hybrid time series SARIMA-SVR approach , 2019, Transportation Research Part E: Logistics and Transportation Review.

[37]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[38]  Germán Hernández,et al.  High-Frequency Trading Strategy Based on Deep Neural Networks , 2016, ICIC.

[39]  Erdogan Dogdu,et al.  An Artificial Neural Network-based Stock Trading System Using Technical Analysis and Big Data Framework , 2017, ACM Southeast Regional Conference.

[40]  Bin Yang,et al.  Correlated Time Series Forecasting using Multi-Task Deep Neural Networks , 2018, CIKM.

[41]  John Moody,et al.  Reinforcement Learning for Trading Systems and Portfolios: Immediate vs Future Rewards , 1998 .

[42]  Andrew Whinston,et al.  Sentiment Manipulation in Online Platforms: An Analysis of Movie Tweets , 2017 .

[43]  Tsan-Ming Choi,et al.  Blockchain-technology-supported platforms for diamond authentication and certification in luxury supply chains , 2019, Transportation Research Part E: Logistics and Transportation Review.

[44]  Ulf Johansson,et al.  High-Frequency Equity Index Futures Trading Using Recurrent Reinforcement Learning with Candlesticks , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[45]  Youyong Kong,et al.  Deep Direct Reinforcement Learning for Financial Signal Representation and Trading , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[47]  Trevor Cohn,et al.  Day trading profit maximization with multi-task learning and technical analysis , 2014, Machine Learning.