Big Data Framework for Finding Patterns in Multi-market Trading Data

In the United States, multimarket trading is becoming very popular for investors, professionals and high-frequency traders. This research focuses on 13 exchanges and applies data mining algorithm, an unsupervised machine learning technique for discovering the relationships between stock exchanges. In this work, we used an association rule (FP-growth) algorithm for finding trading pattern in exchanges. Thirty days NYSE Trade and Quote (TAQ) data were used for these experiments. We implemented a big data framework of Spark clusters on the top of Hadoop to conduct the experiment. The rules and co-relations found in this work seems promising and can be used by the investors and traders to make a decision.

[1]  Taghi M. Khoshgoftaar,et al.  Big Data: Deep Learning for financial sentiment analysis , 2018, Journal of Big Data.

[2]  Bhagwan Chowdhry,et al.  Multimarket Trading and Market Liquidity , 1991 .

[3]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[4]  A. R. Mohamed Shanavas,et al.  Performance Evaluation of Apriori and FP-Growth Algorithms , 2013 .

[5]  Xindong Wu,et al.  Data mining with big data , 2014, IEEE Transactions on Knowledge and Data Engineering.

[6]  Mohsen Kahani,et al.  Semantic association rule mining: A new approach for stock market prediction , 2017, 2017 2nd Conference on Swarm Intelligence and Evolutionary Computation (CSIEC).

[7]  Bin Fang,et al.  Big Data in Finance , 2016 .

[8]  C. Holden,et al.  Liquidity Measurement Problems in Fast, Competitive Markets: Expensive and Cheap Solutions , 2013 .

[9]  Vipin Kumar,et al.  Introduction to Data Mining , 2022, Data Mining and Machine Learning Applications.

[10]  H. Stanley,et al.  Quantifying Trading Behavior in Financial Markets Using Google Trends , 2013, Scientific Reports.

[11]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[12]  Irene Aldridge,et al.  High-frequency Trading High-frequency Trading Industry Strategy Project Engineering Leadership Program , 2022 .

[13]  Jiawei Han,et al.  Frequent pattern mining: current status and future directions , 2007, Data Mining and Knowledge Discovery.

[14]  Tomasz Imielinski,et al.  Mining association rules between sets of items in large databases , 1993, SIGMOD Conference.

[15]  Scott Shenker,et al.  Spark: Cluster Computing with Working Sets , 2010, HotCloud.

[16]  Craig W. Holden,et al.  Liquidity Measurement Problems in Fast, Competitive Markets: Expensive and Cheap Solutions: Liquidity Measurement Problems in Fast, Competitive Markets , 2014 .

[17]  S. Venkatesh,et al.  Emergent Intertransaction Association Rules for Abnormality Detection in Intelligent Environments , 2005, 2005 International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[18]  Srinivasan Parthasarathy,et al.  New Algorithms for Fast Discovery of Association Rules , 1997, KDD.

[19]  Jian Pei,et al.  Mining frequent patterns without candidate generation , 2000, SIGMOD 2000.

[20]  Charu C. Aggarwal,et al.  A Tree Projection Algorithm for Generation of Frequent Item Sets , 2001, J. Parallel Distributed Comput..

[21]  Ramakrishnan Srikant,et al.  Fast algorithms for mining association rules , 1998, VLDB 1998.