RMAIN: Association rules maintenance without reruns through data

Association rules are well recognised as a data mining tool for analysis of transactional data, currently going far beyond the early basket-based applications. A wide spectrum of methods for mining associations have been proposed up to date, including batch and incremental approaches. Most of the accurate incremental methods minimise, but do not completely eliminate reruns through processed data. In this paper we propose a new approximate algorithm RMAIN for incremental maintenance of association rules, which works repeatedly on subsequent portions of new transactions. After a portion has been analysed, the new rules are combined with the old ones, so that no reruns through the processed transactions are performed in the future. The resulting set of rules is kept similar to the one that would be achieved in a batch manner. Unlike other incremental methods, RMAIN is fully separated from a rule mining algorithm and this independence makes it highly general and flexible. Moreover, it operates on rules in their final form, ready for decision support, and not on intermediate representation (frequent itemsets), which requires further processing. These features make the RMAIN algorithm well suited for rule maintenance within knowledge bases of autonomous systems with strongly bounded resources and time for decision making. We evaluated the algorithm on synthetic and real datasets, achieving promising results with respect to either performance or quality of output rules.

[1]  Ming-Syan Chen,et al.  Progressive Partition Miner: An Efficient Algorithm for Mining General Temporal Association Rules , 2003, IEEE Trans. Knowl. Data Eng..

[2]  Ming-Syan Chen,et al.  Sliding window filtering: an efficient method for incremental mining on a time-variant database , 2005, Inf. Syst..

[3]  Heikki Mannila,et al.  Finding interesting rules from large sets of discovered association rules , 1994, CIKM '94.

[4]  Tzung-Pei Hong,et al.  Mining association rules with multiple minimum supports using maximum constraints , 2005, Int. J. Approx. Reason..

[5]  Mohammed J. Zaki Parallel and distributed association mining: a survey , 1999, IEEE Concurr..

[6]  Chien-Ming Chen,et al.  Mining interesting association rules from customer databases and transaction databases , 2004, Inf. Syst..

[7]  Yonatan Aumann,et al.  Borders: An Efficient Algorithm for Association Generation in Dynamic Databases , 1999, Journal of Intelligent Information Systems.

[8]  Louise E. Moser,et al.  Proverbs for Programming in PASCAL , 1986 .

[9]  Philip S. Yu,et al.  Demand-driven frequent itemset mining using pattern structures , 2005, Knowledge and Information Systems.

[10]  Tomasz Imielinski,et al.  Mining association rules between sets of items in large databases , 1993, SIGMOD Conference.

[11]  Christie I. Ezeife,et al.  Mining Incremental Association Rules with Generalized FP-Tree , 2002, Canadian Conference on AI.

[12]  Geert Wets,et al.  Simple association rules (SAR) and the SAR-based rule discovery , 2002 .

[13]  Gerd Stumme,et al.  Generating a Condensed Representation for Association Rules , 2005, Journal of Intelligent Information Systems.

[14]  Yuchang Lu,et al.  Incremental Discovering Association Rules: A Concept Lattice Approach , 1999, PAKDD.

[15]  Won Suk Lee,et al.  Finding recent frequent itemsets adaptively over online data streams , 2003, KDD '03.

[16]  Yannis Manolopoulos,et al.  Memory-adaptive association rules mining , 2004, Inf. Syst..

[17]  Dimitrios Gunopulos,et al.  Constraint-Based Rule Mining in Large, Dense Databases , 2004, Data Mining and Knowledge Discovery.

[18]  Arbee L. P. Chen,et al.  An Efficient Approach for Incremental Association Rule Mining , 1999, PAKDD.

[19]  Jitender S. Deogun,et al.  Sequential Association Rule Mining with Time Lags , 2004, Journal of Intelligent Information Systems.

[20]  Reda Alhajj,et al.  Genetic algorithm based framework for mining fuzzy association rules , 2005, Fuzzy Sets Syst..

[21]  Arbee L. P. Chen,et al.  Efficient Graph-Based Algorithms for Discovering and Maintaining Association Rules in Large Databases , 2001, Knowledge and Information Systems.

[22]  Li Shen,et al.  New Algorithms for Efficient Mining of Association Rules , 1999, Inf. Sci..

[23]  Yuh-Jiuan Tsay,et al.  CBAR: an efficient method for mining association rules , 2005, Knowl. Based Syst..

[24]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[25]  Arbee L. P. Chen,et al.  An efficient approach to discovering knowledge from large databases , 1996, Fourth International Conference on Parallel and Distributed Information Systems.

[26]  Tom Brijs,et al.  Retail market basket analysis: a quantitative modelling approach , 2002 .

[27]  Christian Hidber,et al.  Association Rule Mining , 2017 .

[28]  Sergio A. Alvarez,et al.  Efficient Adaptive-Support Association Rule Mining for Recommender Systems , 2004, Data Mining and Knowledge Discovery.

[29]  Bart Goethals,et al.  Efficient frequent pattern mining , 2002 .

[30]  Bart Goethals,et al.  Proceedings of the IEEE ICDM Workshop on Frequent Itemset Mining Implementations (FIMI 2004) , 2004 .

[31]  Johannes Gehrke,et al.  DEMON: Mining and Monitoring Evolving Data , 2001, IEEE Trans. Knowl. Data Eng..

[32]  Marzena Kryszkiewicz,et al.  Dataless Transitions Between Concise Representations of Frequent Patterns , 2004, Journal of Intelligent Information Systems.

[33]  Aleksander Zgrzywa,et al.  APS: agent's learning with imperfect recall , 2005, 5th International Conference on Intelligent Systems Design and Applications (ISDA'05).

[34]  David Wai-Lok Cheung,et al.  Maintenance of Discovered Knowledge: A Case in Multi-Level Association Rules , 1996, KDD.

[35]  Christie I. Ezeife,et al.  A Low-Scan Incremental Association Rule Maintenance Method Based on the Apriori Property , 2001, Canadian Conference on AI.

[36]  Jennifer Widom,et al.  Models and issues in data stream systems , 2002, PODS.

[37]  Roberto J. Bayardo,et al.  Mining the most interesting rules , 1999, KDD '99.

[38]  Aleksander Zgrzywa,et al.  The Incremental Method for Discovery of Association Rules , 2005, CORES.

[39]  Hong Shen,et al.  Mining Informative Rule Set for Prediction , 2004, Journal of Intelligent Information Systems.

[40]  Damian Dudek,et al.  Using Data Mining Algorithms for Statistical Learning of a Software Agent , 2007, KES-AMSTA.

[41]  Kyuseok Shim,et al.  Mining Optimized Association Rules with Categorical and Numeric Attributes , 2002, IEEE Trans. Knowl. Data Eng..

[42]  Wan-Jui Lee,et al.  Discovery of fuzzy temporal association rules , 2004, IEEE Trans. Syst. Man Cybern. Part B.

[43]  Philip S. Yu,et al.  A New Approach to Online Generation of Association Rules , 2001, IEEE Trans. Knowl. Data Eng..

[44]  Donato Malerba,et al.  Inducing Multi-Level Association Rules from Multiple Relations , 2004, Machine Learning.

[45]  David Wai-Lok Cheung,et al.  Is Sampling Useful in Data Mining? A Case in the Maintenance of Discovered Association Rules , 1998, Data Mining and Knowledge Discovery.

[46]  David Wai-Lok Cheung,et al.  A General Incremental Technique for Maintaining Discovered Association Rules , 1997, DASFAA.

[47]  Frans Coenen,et al.  Tree Structures for Mining Association Rules , 2004, Data Mining and Knowledge Discovery.

[48]  Ramakrishnan Srikant,et al.  Fast Algorithms for Mining Association Rules in Large Databases , 1994, VLDB.

[49]  Chengqi Zhang,et al.  EDUA: An efficient algorithm for dynamic database mining , 2007, Inf. Sci..

[50]  Ran Wolff,et al.  A high-performance distributed algorithm for mining association rules , 2004, Knowledge and Information Systems.

[51]  Mohammed J. Zaki Scalable Algorithms for Association Mining , 2000, IEEE Trans. Knowl. Data Eng..

[52]  Yehuda Lindell,et al.  A Statistical Theory for Quantitative Association Rules , 1999, KDD '99.

[53]  Tzung-Pei Hong,et al.  Flexible online association rule mining based on multidimensional pattern relations , 2006, Inf. Sci..

[54]  Philippe Pucheral,et al.  Bitmap based algorithms for mining association rules , 1998, BDA.