A data mining methodology and its application to semi-automatic knowledge acquisition

We introduce a methodology for knowledge discovery in databases (KDD) where one first discovers large collections of patterns at once, and then performs interactively retrieves subsets of the collection of patterns. The proposed methodology suits such KDD formalisms as association and episode rules, where large collections of potentially interesting rules can be found efficiently. We present methods that support interactive exploration of large collections of rules. With these methods the user can flexibly specify the focus of interest, and also iteratively refine it. We have implemented our methodology in the TASA system which discovers patterns in telecommunication alarm databases. We give concrete examples of how to use frequent patterns in the construction of alarm correlation expert systems.

[1]  Gregory Piatetsky-Shapiro,et al.  Selecting and reporting What Is Interesting , 1996, Advances in Knowledge Discovery and Data Mining.

[2]  Rodney M. Goodman,et al.  NOAA: an expert system managing the telephone network , 1995, Integrated Network Management.

[3]  G. Jakobson,et al.  Alarm correlation , 1993, IEEE Network.

[4]  Heikki Mannila,et al.  Pruning and grouping of discovered association rules , 1995 .

[5]  Heikki Mannila,et al.  Fast Discovery of Association Rules , 1996, Advances in Knowledge Discovery and Data Mining.

[6]  Gregory Piatetsky-Shapiro,et al.  The KDD process for extracting useful knowledge from volumes of data , 1996, CACM.

[7]  Heikki Mannila,et al.  On an algorithm for finding all interesting sentences , 1996 .

[8]  Heikki Mannila,et al.  Discovering Frequent Episodes in Sequences , 1995, KDD.

[9]  Jan M. Zytkow,et al.  From Contingency Tables to Various Forms of Knowledge in Databases , 1996, Advances in Knowledge Discovery and Data Mining.

[10]  Tomasz Imielinski,et al.  Mining association rules between sets of items in large databases , 1993, SIGMOD Conference.

[11]  Heikki Mannila,et al.  Knowledge discovery from telecommunication network alarm databases , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[12]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery: An Overview , 1996, Advances in Knowledge Discovery and Data Mining.

[13]  Ronald J. Brachman,et al.  The Process of Knowledge Discovery in Databases: A First Sketch , 1994, KDD Workshop.

[14]  Heikki Mannila,et al.  Finding interesting rules from large sets of discovered association rules , 1994, CIKM '94.

[15]  Heikki Mannila,et al.  Discovering Generalized Episodes Using Minimal Occurrences , 1996, KDD.