Development of Subjective Measures of Interestingness: From Unexpectedness to Shocking

Knowledge Discovery of Databases (KDD) is the process of extracting previously unknown but useful and significant information from large massive volume of databases. Data Mining is a stage in the entire process of KDD which applies an algorithm to extract interesting patterns. Usually, such algorithms generate huge volume of patterns. These patterns have to be evaluated by using interestingness measures to reflect the user requirements. Interestingness is defined in different ways, (i) Objective measures (ii) Subjective measures. Objective measures such as support and confidence extract meaningful patterns based on the structure of the patterns, while subjective measures such as unexpectedness and novelty reflect the user perspective. In this report, we try to brief the more widely spread and successful subjective measures and propose a new subjective measure of interestingness, i.e. shocking. Keywords—Shocking rules (SHR).

[1]  Abraham Silberschatz,et al.  What Makes Patterns Interesting in Knowledge Discovery Systems , 1996, IEEE Trans. Knowl. Data Eng..

[2]  Jiawei Han,et al.  Profit Mining: From Patterns to Actions , 2002, EDBT.

[3]  Einoshin Suzuki,et al.  Discovery of Surprising Exception Rules Based on Intensity of Implication , 1998, PKDD.

[4]  Balaji Padmanabhan,et al.  Unexpectedness as a Measure of Interestingness in Knowledge Discovery , 1999, Decis. Support Syst..

[5]  Tomasz Imielinski,et al.  Mining association rules between sets of items in large databases , 1993, SIGMOD Conference.

[6]  Sugato Basu and Raymond J. Mooney and Krupakar V. Pasupul Ghosh.  Using Lexical Knowlege to Evaluate the Novelty of Rules Mined from Text , 2001 .

[7]  Hongjun Lu,et al.  Exception Rule Mining with a Relative Interestingness Measure , 2000, PAKDD.

[8]  Wynne Hsu,et al.  Finding Interesting Patterns Using User Expectations , 1999, IEEE Trans. Knowl. Data Eng..

[9]  Zengyou He,et al.  Data Mining for Actionable Knowledge: A Survey , 2005, ArXiv.

[10]  Gregory Piatetsky-Shapiro,et al.  The interestingness of deviations , 1994 .

[11]  Naveen Kumar,et al.  Novelty Framework for Knowledge Discovery in Databases , 2004, DaWaK.

[12]  Abraham Silberschatz,et al.  On Subjective Measures of Interestingness in Knowledge Discovery , 1995, KDD.

[13]  AgrawalRakesh,et al.  Mining association rules between sets of items in large databases , 1993 .

[14]  Balaji Padmanabhan,et al.  A Belief-Driven Method for Discovering Unexpected Patterns , 1998, KDD.

[15]  Wynne Hsu,et al.  Using General Impressions to Analyze Discovered Classification Rules , 1997, KDD.

[16]  Alex Berson,et al.  Data Warehousing, Data Mining, and OLAP , 1997 .

[17]  Wynne Hsu,et al.  Identifying Interesting Missing Patterns , 1997 .