Feature Selection for Forecasting

This work investigates the importance of feature selection for improving the forecasting performance of machine learning algorithms for financial data. Artificial neural networks (ANN), convolutional neural networks (CNN), long-short term memory (LSTM) networks, as well as linear models were applied for forecasting purposes. The Feature Selection with Annealing (FSA) algorithm was used to select the features from about 1000 possible predictors obtained from 26 technical indicators with specific periods and their lags. In addition to this, the Boruta feature selection algorithm was applied as a baseline feature selection method. The dependent variables consisted of daily logarithmic returns and daily trends of ten financial data sets, including cryptocurrency and different stocks. Experiments indicate that the FSA algorithm increased the performance of ML models regardless of the problem type. The FSA hybrid machine learning models showed better performance in 10 out of 10 data sets for regression and 8 out of 10 data sets for classification. None of the hybrid Boruta models outperformed the hybrid FSA models. However, the BORCNN model performance was comparable to the best model for 4 out of 10 data sets for regression estimates. BOR-LR and BOR-CNN models showed comparable performance with the best hybrid FSA models in 2 out of 10 datasets for classification. FSA was observed to improve the model performance in both better performance metrics as well as a decreased computation time by providing a lower dimensional input feature space.

[1]  T. Chaudhuri,et al.  A hybrid approach to forecasting futures prices with simultaneous consideration of optimality in ensemble feature selection and advanced artificial intelligence , 2022, Technological Forecasting and Social Change.

[2]  Perry Sadorsky,et al.  Forecasting Bitcoin price direction with random forests: How important are interest rates, inflation, and market volatility? , 2022, Machine Learning with Applications.

[3]  Hao Wang,et al.  Forecasting cryptocurrency price using convolutional neural networks with weighted and attentive memory channels , 2021, Expert Syst. Appl..

[4]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[5]  Ugo Fiore,et al.  Forecasting directional movement of Forex data using LSTM with technical and macroeconomic indicators , 2021, Financial Innovation.

[6]  Sebastián Maldonado,et al.  SVR-FFS: A novel forward feature selection approach for high-frequency time series forecasting using support vector regression , 2020, Expert Syst. Appl..

[7]  Jianzhou Wang,et al.  Developing a deep learning framework with two-stage feature selection for multivariate financial time series forecasting , 2020, Expert Syst. Appl..

[8]  Tomé Almeida Borges,et al.  Ensemble of machine learning algorithms for cryptocurrency investment with different data resampling methods , 2020, Appl. Soft Comput..

[9]  Erdinc Akyildirim,et al.  Prediction of cryptocurrency returns using machine learning , 2020, Annals of Operations Research.

[10]  Zheshi Chen,et al.  Bitcoin price prediction using machine learning: An approach to sample dimension engineering , 2020, J. Comput. Appl. Math..

[11]  Sun-Hak Kim,et al.  Using Genetic Algorithm and NARX Neural Network to Forecast Daily Bitcoin Price , 2019, Computational Economics.

[12]  Constantin Zopounidis,et al.  Bitcoin price forecasting with neuro-fuzzy techniques , 2019, Eur. J. Oper. Res..

[13]  Herbert Kimura,et al.  Literature review: Machine learning techniques applied to financial market prediction , 2019, Expert Syst. Appl..

[14]  Haruna Isah,et al.  Stock Market Analysis: A Review and Taxonomy of Prediction Techniques , 2019, International Journal of Financial Studies.

[15]  Nico Smuts,et al.  What Drives Cryptocurrency Prices? , 2019, SIGMETRICS Perform. Evaluation Rev..

[16]  Vivek Tiwari,et al.  Stock Market Price Prediction Using LSTM RNN , 2018, Advances in Intelligent Systems and Computing.

[17]  Alexandros Iosifidis,et al.  Machine Learning for Forecasting Mid-Price Movements Using Limit Order Book Data , 2018, IEEE Access.

[18]  Simon Caton,et al.  Predicting the Price of Bitcoin Using Machine Learning , 2018, 2018 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP).

[19]  Ruonan Rao,et al.  Time-Weighted LSTM Model with Redefined Labeling for Stock Trend Prediction , 2017, 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI).

[20]  Georgia Papacharalampous,et al.  Variable Selection in Time Series Forecasting Using Random Forests , 2017, Algorithms.

[21]  Alexandros Iosifidis,et al.  Using deep learning to detect price change indications in financial markets , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[22]  Yulei Rao,et al.  A deep learning framework for financial time series using stacked autoencoders and long-short term memory , 2017, PloS one.

[23]  Liang-Ying Wei,et al.  A hybrid ANFIS model based on empirical mode decomposition for stock time series forecasting , 2016, Appl. Soft Comput..

[24]  Michel Ballings,et al.  Evaluating multiple classifiers for stock price direction prediction , 2015, Expert Syst. Appl..

[25]  Sahil Shah,et al.  Predicting stock market index using fusion of machine learning techniques , 2015, Expert Syst. Appl..

[26]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[27]  Adrian Barbu,et al.  Feature Selection with Annealing for Computer Vision and Big Data Learning , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Alex Graves.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[29]  Ömer Kaan Baykan,et al.  Predicting direction of stock price index movement using artificial neural networks and support vector machines: The sample of the Istanbul Stock Exchange , 2011, Expert Syst. Appl..

[30]  Witold R. Rudnicki,et al.  Feature Selection with the Boruta Package , 2010 .

[31]  Sheng-Hsun Hsu,et al.  A two-stage architecture for stock price forecasting by integrating self-organizing map and support vector regression , 2009, Expert Syst. Appl..

[32]  Baikunth Nath,et al.  A fusion model of HMM, ANN and GA for stock market forecasting , 2007, Expert Syst. Appl..

[33]  Manish Kumar,et al.  Forecasting Stock Index Movement: A Comparison of Support Vector Machines and Random Forest , 2006 .

[34]  Shouyang Wang,et al.  Forecasting stock market movement direction with support vector machine , 2005, Comput. Oper. Res..

[35]  D. O. Hebb,et al.  The First Stage of Perception: Growth of the Assembly , 2005 .

[36]  Dennis Olson,et al.  Neural network forecasts of Canadian stock returns using accounting ratios , 2003 .

[37]  M. Leung,et al.  Forecasting Stock Indices: A Comparison of Classification and Level Estimation Models , 1999 .

[38]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[39]  Kazuo Asakawa,et al.  Stock market prediction system with modular neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[40]  Ken'ichi Kamijo,et al.  Stock price pattern recognition-a recurrent neural network approach , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[41]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[42]  E. Fama.  EFFICIENT CAPITAL MARKETS: A REVIEW OF THEORY AND EMPIRICAL WORK* , 1970 .

[43]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[44]  Leslie Kanthan,et al.  Ascertaining price formation in cryptocurrency markets with machine learning , 2021, The European Journal of Finance.

[45]  Yong Xiao,et al.  Dynamic Multi-Scale Convolutional Neural Network for Time Series Classification , 2020, IEEE Access.

[46]  Mingxi Liu,et al.  A novel cryptocurrency price trend forecasting model based on LightGBM , 2020 .

[47]  Salim Lahmiri,et al.  Cryptocurrency forecasting with deep learning chaotic neural networks , 2019, Chaos, Solitons & Fractals.

[48]  Luca Di Persio,et al.  Recurrent Neural Networks Approach to the Financial Forecast of Google Assets , 2017 .

[49]  Harshal Patel,et al.  Predicting Stock Prices Using LSTM , 2017 .

[50]  Sahil Shah,et al.  Predicting stock and stock price index movement using Trend Deterministic Data Preparation and machine learning techniques , 2015, Expert Syst. Appl..

[51]  Marvin Minsky,et al.  An introduction to computational geometry , 1969 .