Effect of dimensionality reduction on stock selection with cluster analysis in different market situations

Abstract Dimensionality reduction is inevitable in stock selection with cluster analysis. Considering relations among dimensionality reduction, noise trading, and market situations, we empirically investigate the effect of dimensionality-reduction methods–principal component analysis, stacked autoencoder, and stacked restricted Boltzmann machine–on stock selection with cluster analysis in different market situations. Based on the index fluctuation, the market is divided into sideways and trend situations. For the CSI 100 and Nikkei 225 constituent stocks, experimental results show that: (1) In sideways situations, dimensionality reduction hardly improves the performance of stock selection with cluster analysis; (2) the advantage of dimensionality reduction is mainly reflected in trend situations, but whether it is in an up or down trend depends on the market analyzed. More importantly, according to the above findings and assuming that the dimensionality-reduction effect will continue, we propose a rotation strategy with and without dimensionality reduction. The results of experiments show that the proposed rotation strategy outperforms the stock market indices as well as the stock-selection strategies based on dimensionality reduction and cluster analysis. These findings offer practical insights into how dimensionality reduction can be efficiently used for stock selection.

[1]  V. Saravanan,et al.  An Increased Performance of Clustering High Dimensional Data Using Principal Component Analysis , 2010, 2010 First International Conference on Integrated Intelligent Computing.

[2]  M. K. Tiwari,et al.  Clustering Indian stock market data for portfolio management , 2010, Expert Syst. Appl..

[3]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[4]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[5]  Benjamin Miranda Tabak,et al.  Topological properties of stock market networks: The case of Brazil , 2010 .

[6]  E. Fama,et al.  The Cross‐Section of Expected Stock Returns , 1992 .

[7]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[8]  S. Titman,et al.  Momentum Investment Strategies, Portfolio Performance, and Herding: A Study of Mutual Fund Behavior , 1994 .

[9]  E. Fama,et al.  International Tests of a Five-Factor Asset Pricing Model , 2015 .

[10]  Hongyuan Zha,et al.  Adaptive dimension reduction for clustering high dimensional data , 2002 .

[11]  Juan Gabriel Brida,et al.  Hierarchical Structure of the German Stock Market , 2007, Expert Syst. Appl..

[12]  Parvez Ahmed,et al.  Multistyle Rotation Strategies , 2002 .

[13]  Alan Hanna,et al.  A top-down approach to identifying bull and bear market states , 2018 .

[14]  Huan Liu,et al.  Subspace clustering for high dimensional data: a review , 2004, SKDD.

[15]  Jun Li,et al.  Investor Attention, Psychological Anchors, and Stock Return Predictability , 2010 .

[16]  Manuel Mendoza,et al.  Dynamic Stock Selection Strategies: A Structured Factor Model Framework , 2012 .

[17]  Rahul Verma,et al.  Noise trading and stock market volatility , 2007 .

[18]  G. Phillips,et al.  Low- (Economic) Volatility Investing , 2012, The Journal of Wealth Management.

[19]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[20]  J. Hsu,et al.  Low-Volatility Investing , 2013, The Journal of Index Investing.

[21]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[22]  Jianguo Liu,et al.  Deep Stock Representation Learning: From Candlestick Charts to Investment Decisions , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[23]  Vipin Kumar,et al.  The Challenges of Clustering High Dimensional Data , 2004 .

[24]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[25]  Tian Qiu,et al.  Dynamic Portfolio Strategy Using Clustering Approach , 2016, PloS one.

[26]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[27]  Jan Hendrik Witte,et al.  Deep Learning for Finance: Deep Portfolios , 2016 .

[28]  Michael J. Cooper,et al.  Market States and Momentum , 2004 .

[29]  James B. Heian,et al.  DIFFERENCES OF OPINION, OVERCONFIDENCE, AND THE HIGH‐VOLUME PREMIUM , 2011 .

[30]  Jorge Cadima,et al.  Principal component analysis: a review and recent developments , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[31]  Andre Lucas,et al.  Stock selection, style rotation, and risk , 2002 .

[32]  Chulwoo Han,et al.  Deep learning networks for stock market analysis and prediction: Methodology, data representations, and case studies , 2017, Expert Syst. Appl..

[33]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[34]  Lim Kai Jie Shawn,et al.  Return and Risk-Return Ratio Based Momentum Strategies: A Fresh Perspective , 2012 .

[35]  Jatinderkumar R. Saini,et al.  Agent based Stock Clustering for Efficient Portfolio Management , 2015 .

[36]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[37]  Sahil Shah,et al.  Predicting stock and stock price index movement using Trend Deterministic Data Preparation and machine learning techniques , 2015, Expert Syst. Appl..

[38]  Carmela Iorio,et al.  A P-spline based clustering approach for portfolio selection , 2018, Expert Syst. Appl..

[39]  Xiaola Lin,et al.  Feature extraction using Restricted Boltzmann Machine for stock price prediction , 2012, 2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE).

[40]  Silvano Cincotti,et al.  Clustering of financial time series with application to index and enhanced index tracking portfolio , 2005 .

[41]  Bruno Silva,et al.  Feature Clustering with Self-organizing Maps and an Application to Financial Time-series for Portfolio Selection , 2010, IJCCI.

[42]  Qi Tian,et al.  Feature representation for statistical-learning-based object detection: A review , 2015, Pattern Recognit..

[43]  Sergio Da Silva,et al.  Stock selection based on cluster analysis , 2005 .

[44]  Michel Verleysen,et al.  The Curse of Dimensionality in Data Mining and Time Series Prediction , 2005, IWANN.