Transformer-Based Deep Learning Model for Stock Price Prediction: A Case Study on Bangladesh Stock Market

In the modern capital market, the price of a stock is often considered to be highly volatile and unpredictable because of various social, financial, political and other dynamic factors. With calculated and thoughtful investment, stock market can ensure a handsome profit with minimal capital investment, while incorrect prediction can easily bring catastrophic financial loss to the investors. This paper introduces the application of a recently introduced machine learning model — the transformer model, to predict the future price of stocks of Dhaka Stock Exchange (DSE), the leading stock exchange in Bangladesh. The transformer model has been widely leveraged for natural language processing and computer vision tasks, but, to the best of our knowledge, has never been used for stock price prediction task task using DSE data. Recently, the introduction of time2vec encoding to represent the time series features has made it possible to employ the transformer model for the stock price prediction. This paper aims to leverage these two effective techniques to discover forecasting ability on the volatile stock market of DSE. We deal with the historical daily and weekly data of eight specific stocks listed in DSE. Our experiments demonstrate promising results and acceptable root-mean-squared error on most of the stocks. We also compare the performance of our model with that of a well-known benchmark stock forecasting model called ARIMA and report satisfactory results.

[1]  Eduardo Ramos-Pérez,et al.  Multi-Transformer: A New Neural Network-Based Architecture for Forecasting S&P Volatility , 2021, Mathematics.

[2]  Gary Wills,et al.  A Survey on Machine Learning for Stock Price Prediction: Algorithms and Techniques , 2020, FEMIB.

[3]  Changjun Zhou,et al.  Forecasting stock prices with long-short term memory neural network based on attention mechanism , 2020, PloS one.

[4]  Sanjay Thakur,et al.  Time2Vec: Learning a Vector Representation of Time , 2019, ArXiv.

[5]  Ha Young Kim,et al.  Forecasting stock prices with a feature fusion LSTM-CNN model using different representations of the same data , 2019, PloS one.

[6]  Md. Mujibur Rahman Majumder,et al.  Indices prediction of Bangladeshi stock by using time series forecasting and performance analysis , 2019, 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE).

[7]  Sheng Chen,et al.  Stock Prediction Using Convolutional Neural Network , 2018, IOP Conference Series: Materials Science and Engineering.

[8]  Weina Wang,et al.  A big data framework for stock price forecasting using fuzzy time series , 2017, Multimedia Tools and Applications.

[9]  A. Murat Ozbayoglu,et al.  A deep learning based stock trading model with 2-D CNN trend detection , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[10]  S. Khajavi,et al.  Prediction of Stock Price using Particle Swarm Optimization Algorithm and Box-Jenkins Time Series , 2017 .

[11]  K. P. Soman,et al.  Stock price prediction using LSTM, RNN and CNN-sliding window model , 2017, 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[12]  Alexandros Iosifidis,et al.  Forecasting Stock Prices from the Limit Order Book Using Convolutional Neural Networks , 2017, 2017 IEEE 19th Conference on Business Informatics (CBI).

[13]  Debahuti Mishra,et al.  A Survey on Impact of Bio-inspired Computation on Stock Market Prediction , 2017 .

[14]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[15]  Barack Wamkaya Wanjawa,et al.  ANN Model to Predict Stock Prices at Stock Exchange Markets , 2014, ArXiv.

[16]  Eberhard Schöneburg,et al.  Stock price prediction using neural networks : A project report , 2003 .

[17]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[18]  Hongfei Lin,et al.  Transformer-Based Capsule Network For Stock Movement Prediction , 2019 .

[19]  K. P. Soman,et al.  NSE Stock Market Prediction Using Deep-Learning Models , 2018 .

[20]  Md. Matiar Rahman,et al.  Modeling and predicting stock market returns: A case study on dhaka stock exchange of Bangladesh , 2017 .