A user parameter-free approach for mining robust sequential classification rules

Sequential data are generated in many domains of science and technology. Although many studies have been carried out for sequence classification in the past decade, the problem is still a challenge, particularly for pattern-based methods. We identify two important issues related to pattern-based sequence classification, which motivate the present work: the curse of parameter tuning and the instability of common interestingness measures. To alleviate these issues, we suggest a new approach and framework for mining sequential rule patterns for classification purpose. We introduce a space of rule pattern models and a prior distribution defined on this model space. From this model space, we define a Bayesian criterion for evaluating the interest of sequential patterns. We also develop a user parameter-free algorithm to efficiently mine sequential patterns from the model space. Extensive experiments show that (i) the new criterion identifies interesting and robust patterns, (ii) the direct use of the mined rules as new features in a classification process demonstrates higher inductive performance than the state-of-the-art sequential pattern-based classifiers.

[1]  Heikki Mannila,et al.  Multiple Uses of Frequent Sets and Condensed Representations (Extended Abstract) , 1996, KDD.

[2]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .

[3]  Frans Coenen,et al.  The effect of threshold values on association rule based classification accuracy , 2007, Data Knowl. Eng..

[4]  George Karypis,et al.  Evaluation of Techniques for Classifying Biological Sequences , 2002, PAKDD.

[5]  Marc Boullé,et al.  MODL: A Bayes optimal discretization method for continuous attributes , 2006, Machine Learning.

[6]  Henrik Grosskreutz,et al.  A Relevance Criterion for Sequential Patterns , 2013, ECML/PKDD.

[7]  Jilles Vreeken,et al.  The long and the short of it: summarising event sequences with serial episodes , 2012, KDD.

[8]  Osmar R. Zaïane,et al.  An Occurrence Based Approach to Mine Emerging Sequences , 2010, DaWak.

[9]  Claude E. Shannon,et al.  The Mathematical Theory of Communication , 1950 .

[10]  Jean-François Boulicaut,et al.  Feature Construction Based on Closedness Properties Is Not That Simple , 2008, PAKDD.

[11]  Toon Calders,et al.  Mining Compressing Sequential Patterns , 2012, Stat. Anal. Data Min..

[12]  Elena Baralis,et al.  Compact Representations of Sequential Classification Rules , 2008, Data Mining: Foundations and Practice.

[13]  Vincent S. Tseng,et al.  CBS: A New Classification Method by Using Sequential Patterns , 2005, SDM.

[14]  Ke Wang,et al.  Frequent-subsequence-based prediction of outer membrane proteins , 2003, KDD '03.

[15]  Jerome L. Myers,et al.  Research Design and Statistical Analysis , 1991 .

[16]  Marc Boullé,et al.  Compression-Based Averaging of Selective Naive Bayes Classifiers , 2007, J. Mach. Learn. Res..

[17]  Nada Lavrac,et al.  A Study of Relevance for Learning in Deductive Databases , 1999, J. Log. Program..

[18]  Jason Weston,et al.  Mismatch String Kernels for SVM Protein Classification , 2002, NIPS.

[19]  Mohammed J. Zaki,et al.  Learning sequential classifiers from long and noisy discrete-event sequences efficiently , 2014, Data Mining and Knowledge Discovery.

[20]  Jiawei Han,et al.  BIDE: efficient mining of frequent closed sequences , 2004, Proceedings. 20th International Conference on Data Engineering.

[21]  Mohammed J. Zaki,et al.  Mining features for sequence classification , 1999, KDD '99.

[22]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[23]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[24]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[25]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[26]  Alípio Mário Jorge,et al.  Distribution Rules with Numeric Attributes of Interest , 2006, PKDD.

[27]  Ana Margarida de Jesus,et al.  Improving Methods for Single-label Text Categorization , 2007 .

[28]  Nello Cristianini,et al.  Classification using String Kernels , 2000 .

[29]  Johannes Gehrke,et al.  Sequential PAttern mining using a bitmap representation , 2002, KDD.

[30]  Mohammed J. Zaki Sequence mining in categorical domains: incorporating constraints , 2000, CIKM '00.

[31]  Ming Li,et al.  Minimum description length induction, Bayesianism, and Kolmogorov complexity , 1999, IEEE Trans. Inf. Theory.

[32]  Chedy Raïssi,et al.  Sequence Classification Based on Delta-Free Sequential Patterns , 2014, 2014 IEEE International Conference on Data Mining.

[33]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[34]  Christopher D. Carothers,et al.  VOGUE: A variable order hidden Markov model with duration based on frequent sequence mining , 2010, TKDD.

[35]  Mark A. Pitt,et al.  Advances in Minimum Description Length: Theory and Applications , 2005 .

[36]  Philip S. Yu,et al.  Direct mining of discriminative and essential frequent patterns via model-based search tree , 2008, KDD.

[37]  Wynne Hsu,et al.  Integrating Classification and Association Rule Mining , 1998, KDD.

[38]  Chedy Raïssi,et al.  On measuring similarity for sequences of itemsets , 2014, Data Mining and Knowledge Discovery.

[39]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[40]  James Bailey,et al.  Mining minimal distinguishing subsequence patterns with gap constraints , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[41]  Marc Boullé,et al.  A Parameter-Free Approach for Mining Robust Sequential Classification Rules , 2015, 2015 IEEE International Conference on Data Mining.

[42]  Marc Boullé,et al.  A Bayesian Approach for Classification Rule Mining in Quantitative Databases , 2012, ECML/PKDD.

[43]  Jian Pei,et al.  A brief survey on sequence classification , 2010, SKDD.

[44]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[45]  Emmanuel Viennet,et al.  bitSPADE: A Lattice-based Sequential Pattern Mining Algorithm Using Bitmap Representation , 2006, Sixth International Conference on Data Mining (ICDM'06).

[46]  Fabian Mörchen,et al.  Efficient mining of understandable patterns from multivariate interval time series , 2007, Data Mining and Knowledge Discovery.

[47]  Jiawei Han,et al.  Discriminative Frequent Pattern Analysis for Effective Classification , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[48]  Vipin Kumar,et al.  Discovery of Web Robot Sessions Based on their Navigational Patterns , 2004, Data Mining and Knowledge Discovery.

[49]  Ramakrishnan Srikant,et al.  Mining sequential patterns , 1995, Proceedings of the Eleventh International Conference on Data Engineering.

[50]  Siegfried Nijssen,et al.  Supervised Pattern Mining and Applications to Classification , 2014, Frequent Pattern Mining.

[51]  Boris Cule,et al.  Itemset Based Sequence Classification , 2013, ECML/PKDD.

[52]  Dmitriy Fradkin,et al.  Under Consideration for Publication in Knowledge and Information Systems Mining Sequential Patterns for Classification , 2022 .