On Characterization and Discovery of Minimal Unexpected Patterns in Data Mining Applications

A drawback of traditional data mining methods is that they do not leverage prior knowledge of users. In many business settings, managers and analysts have significant intuition based on several years of experience. In prior work we proposed a method that could discover unexpected patterns in data by using this domain knowledge in a systematic manner. In this paper we continue our focus on discovering unexpected patterns and propose new methods for discovering a minimal set of unexpected patterns that discover orders of magnitude fewer patterns and yet retain most of the truly unexpected ones. We demonstrate the strengths of this approach experimentally using a case study application in a marketing domain.

[1]  Tom M. Mitchell,et al.  Generalization as Search , 2002 .

[2]  Patrick Henry Winston,et al.  Learning structural descriptions from examples , 1970 .

[3]  Steven A. Vere,et al.  Inductive learning of relational productions , 1977, SGAR.

[4]  Gediminas Adomavicius,et al.  Discovery of Actionable Patterns in Databases: the Action Hierarchy Approach , 1997, KDD.

[5]  Wynne Hsu,et al.  Using General Impressions to Analyze Discovered Classification Rules , 1997, KDD.

[6]  Rajeev Motwani,et al.  Dynamic itemset counting and implication rules for market basket data , 1997, SIGMOD '97.

[7]  Alexander Tuzhilin,et al.  Discovering Unexpected Patterns in Temporal Data Using Temporal Logic , 1997, Temporal Databases, Dagstuhl.

[8]  Balaji Padmanabhan,et al.  Unexpectedness as a Measure of Interestingness in Knowledge Discovery , 1999, Decis. Support Syst..

[9]  Roberto J. Bayardo,et al.  Mining the most interesting rules , 1999, KDD '99.

[10]  Ramesh Subramonian.  Defining diff as a Data Mining Primitive , 1998, KDD.

[11]  Balaji Padmanabhan,et al.  A Belief-Driven Method for Discovering Unexpected Patterns , 1998, KDD.

[12]  Ramakrishnan Srikant,et al.  Mining quantitative association rules in large relational tables , 1996, SIGMOD '96.

[13]  Nie Yong.  Mining quantitative association rules , 2000 .

[14]  Laks V. S. Lakshmanan,et al.  Interestingness and Pruning of Mined Patterns , 1999, 1999 ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery.

[15]  Wynne Hsu,et al.  Post-Analysis of Learned Rules , 1996, AAAI/IAAI, Vol. 1.

[16]  Heikki Mannila,et al.  Finding interesting rules from large sets of discovered association rules , 1994, CIKM '94.

[17]  Ramakrishnan Srikant,et al.  Mining Association Rules with Item Constraints , 1997, KDD.

[18]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery: An Overview , 1996, Advances in Knowledge Discovery and Data Mining.

[19]  Sunita Sarawagi,et al.  Mining Surprising Patterns Using Temporal Description Length , 1998, VLDB.

[20]  Tom M. Mitchell,et al.  Version Spaces: A Candidate Elimination Approach to Rule Learning , 1977, IJCAI.

[21]  Gregory Piatetsky,et al.  Selecting and Reporting What is Interesting � The KEFIR Application to Healthcare Data , 2004 .

[22]  Frederick Hayes-Roth,et al.  An Automatically Compilable Recognition Network For Structured Patterns , 1975, IJCAI.

[23]  Tomasz Imielinski,et al.  Mining association rules between sets of items in large databases , 1993, SIGMOD Conference.

[24]  Abraham Silberschatz,et al.  What Makes Patterns Interesting in Knowledge Discovery Systems , 1996, IEEE Trans. Knowl. Data Eng..

[25]  Heikki Mannila,et al.  Pruning and grouping of discovered association rules , 1995 .

[26]  Heikki Mannila,et al.  Fast Discovery of Association Rules , 1996, Advances in Knowledge Discovery and Data Mining.

[27]  Gregory Piatetsky-Shapiro,et al.  The interestingness of deviations , 1994 .

[28]  Einoshin Suzuki,et al.  Autonomous Discovery of Reliable Exception Rules , 1997, KDD.

[29]  Abraham Silberschatz,et al.  On Subjective Measures of Interestingness in Knowledge Discovery , 1995, KDD.

[30]  Tom M. Mitchell,et al.  MODEL-DIRECTED LEARNING OF PRODUCTION RULES1 , 1978 .

[31]  Dimitrios Gunopulos,et al.  Constraint-Based Rule Mining in Large, Dense Databases , 2004, Data Mining and Knowledge Discovery.