Efficient adaptive frequent pattern mining techniques for market analysis in sequential and parallel systems

The classical applications of Association Rule Mining (ARM) are market analysis, network traffic analysis, and web log analysis where strategic decisions are made by analyzing the frequent itemsets from a large pool of data. Datasets in such domains are constantly updated and as they require an efficient Frequent Pattern Mining (FPM) algorithm which is capable of extracting the required information. Several incremental algorithms have been proposed to generate frequent patterns, but they are ineffective with very large datasets and do not provide the user interaction to adjust the minimum support value. This paper first presents an efficient interactive sequential FPM algorithm that uses the knowledge gained in the previous mining steps to incrementally mine the updated database with fewer complexities. Then to further reduce the time complexity it proposes an efficient interactive and incremental parallel mining algorithm. It also prepares incremental frequent patterns, without generating local frequent itemsets with less communication and synchronization overheads.

[1]  Philip S. Yu,et al.  Efficient parallel data mining for association rules , 1995, CIKM '95.

[2]  Wenguang Chen,et al.  Tree partition based parallel frequent pattern mining on shared memory systems , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[3]  David Wai-Lok Cheung,et al.  Efficient Mining of Association Rules in Distributed Databases , 1996, IEEE Trans. Knowl. Data Eng..

[4]  Philip S. Yu,et al.  An effective hash-based algorithm for mining association rules , 1995, SIGMOD '95.

[5]  Don-Lin Yang,et al.  IMBT--A Binary Tree for Efficient Support Counting of Incremental Data Mining , 2009, 2009 International Conference on Computational Science and Engineering.

[6]  Ranjit Biswas,et al.  YAMI: Incremental Mining of Interesting Association Patterns , 2012, Int. Arab J. Inf. Technol..

[7]  Shraddha Savaliya,et al.  An Effective Hash-Based Algorithm for Mining Association Rules , 2015 .

[8]  Shamkant B. Navathe,et al.  An Efficient Algorithm for Mining Association Rules in Large Databases , 1995, VLDB.

[9]  Ketan Shah,et al.  An Efficient Parallel Approach for Frequent Itemset Mining of Incremental Data , 2012 .

[10]  Zhan Li,et al.  Knowledge and Information Systems , 2007 .

[11]  Rajeev Motwani,et al.  Dynamic itemset counting and implication rules for market basket data , 1997, SIGMOD '97.

[12]  Mohammed J. Zaki,et al.  Incremental and interactive mining for frequent itemsets in evolving databases , 2003 .

[13]  Rakesh Agrawal,et al.  Parallel Mining of Association Rules: Design, Implementation and Experience , 1999 .

[14]  Mohammad Nadimi,et al.  An Efficient Prefix Tree for Incremental Frequent Pattern Mining , 2013 .

[15]  Jian Pei,et al.  Mining frequent patterns without candidate generation , 2000, SIGMOD '00.

[16]  R. Nedunchezhian,et al.  IAPI QUAD-FILTER: AN INTERACTIVE AND ADAPTIVE PARTITIONED APPROACH FOR INCREMENTAL FREQUENT PATTERN MINING , 2014 .

[17]  Rakesh Agarwal,et al.  Fast Algorithms for Mining Association Rules , 1994, VLDB 1994.

[18]  Christian Hidber,et al.  Association Rule Mining , 2017 .

[19]  Qing He,et al.  Parallel Implementation of Apriori Algorithm Based on MapReduce , 2012, 2012 13th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing.

[20]  Young-Koo Lee,et al.  Efficient single-pass frequent pattern mining using a prefix-tree , 2009, Inf. Sci..