Efficient model selection for predictive pattern mining model by safe pattern pruning

Predictive pattern mining is an approach used to construct prediction models when the input is represented by structured data, such as sets, graphs, and sequences. The main idea behind predictive pattern mining is to build a prediction model by considering substructures, such as subsets, subgraphs, and subsequences (referred to as patterns), present in the structured data as features of the model. The primary challenge in predictive pattern mining lies in the exponential growth of the number of patterns with the complexity of the structured data. In this study, we propose the Safe Pattern Pruning (SPP) method to address the explosion of pattern numbers in predictive pattern mining. We also discuss how it can be effectively employed throughout the entire model building process in practical data analysis. To demonstrate the effectiveness of the proposed method, we conduct numerical experiments on regression and classification problems involving sets, graphs, and sequences.

[1]  I. Takeuchi,et al.  Safe RuleFit: Learning Optimal Sparse Rule Model by Meta Safe Screening , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Yasuo Tabei,et al.  Stat-DSM: Statistically Discriminative Sub-Trajectory Mining With Multiple Testing Correction , 2022, IEEE Transactions on Knowledge and Data Engineering.

[3]  Ichiro Takeuchi,et al.  Fast and More Powerful Selective Inference for Sparse High-order Interaction Model , 2021, AAAI.

[4]  Ichiro Takeuchi,et al.  Supervised sequential pattern mining of event sequences in sport to identify important patterns of play: An application to rugby union , 2020, PloS one.

[5]  Ichiro Takeuchi,et al.  Learning Interpretable Metric between Graphs: Convex Formulation and Computation with Graph Mining , 2019, KDD.

[6]  Rahul Mazumder,et al.  Learning Hierarchical Interactions at Scale: A Convex Optimization Approach , 2019, AISTATS.

[7]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[8]  Yasuo Tabei,et al.  Entire Regularization Path for Sparse Nonnegative Interaction Model , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[9]  Ichiro Takeuchi,et al.  Safe Grid Search with Optimal Complexity , 2018, ICML.

[10]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[11]  Yee Whye Teh,et al.  Set Transformer , 2018, ICML.

[12]  Jun Sakuma,et al.  Efficiently Monitoring Small Data Modification Effect for Large-Scale Learning in Changing Environment , 2018, AAAI.

[13]  Jean-Philippe Vert,et al.  WHInter: A Working set algorithm for High-dimensional sparse second order Interaction models , 2018, ICML.

[14]  Ichiro Takeuchi,et al.  Safe Triplet Screening for Distance Metric Learning , 2018, Neural Computation.

[15]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[16]  Ichiro Takeuchi,et al.  Selective Inference for Sparse High-Order Interaction Models , 2017, ICML.

[17]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[18]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Alexandre Gramfort,et al.  Gap Safe screening rules for sparsity enforcing penalties , 2016, J. Mach. Learn. Res..

[20]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[21]  Ichiro Takeuchi,et al.  Safe Pattern Pruning: An Efficient Approach for Predictive Pattern Mining , 2016, KDD.

[22]  Jun Sakuma,et al.  Secure Approximation Guarantee for Cryptographically Private Empirical Risk Minimization , 2016, ACML.

[23]  Ichiro Takeuchi,et al.  Simultaneous Safe Screening of Features and Samples in Doubly Sparse Modeling , 2016, ICML.

[24]  Alexandre Gramfort,et al.  Mind the duality gap: safer rules for the Lasso , 2015, ICML.

[25]  Ichiro Takeuchi,et al.  Quick Sensitivity Analysis for Incremental Data Modification and Its Application to Leave-one-out CV in Linear Classification Problems , 2015, KDD.

[26]  Ichiro Takeuchi,et al.  Regularization Path of Cross-Validation Error Lower Bounds , 2015, NIPS.

[27]  Rémi Gribonval,et al.  Dynamic Screening: Accelerating First-Order Algorithms for the Lasso and Group-Lasso , 2014, IEEE Transactions on Signal Processing.

[28]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[29]  Peter J. Ramadge,et al.  Screening Tests for Lasso Problems , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Ichiro Takeuchi,et al.  Discovering combinatorial interactions in survival data , 2013, Bioinform..

[31]  Alex Graves.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[32]  Ichiro Takeuchi,et al.  Safe Screening of Non-Support Vectors in Pathwise SVM Computation , 2013, ICML.

[33]  Jie Wang,et al.  Lasso screening rules via dual polytope projection , 2012, J. Mach. Learn. Res..

[34]  Laurent El Ghaoui,et al.  Safe Feature Elimination for the LASSO and Sparse Supervised Learning Problems , 2010, 1009.4219.

[35]  Kurt Mehlhorn,et al.  Efficient graphlet kernels for large graph comparison , 2009, AISTATS.

[36]  Sebastian Nowozin,et al.  gBoost: a mathematical programming approach to graph classification and regression , 2009, Machine Learning.

[37]  Paul Tseng,et al.  A coordinate gradient descent method for nonsmooth separable minimization , 2008, Math. Program..

[38]  Karsten M. Borgwardt,et al.  Graph Kernels , 2008, J. Mach. Learn. Res..

[39]  Takeaki Uno,et al.  Mining complex genotypic features for predicting HIV-1 drug resistance , 2007, Bioinform..

[40]  Hiroto Saigo,et al.  A Linear Programming Approach for Molecular QSAR analysis , 2006 .

[41]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[42]  Hisashi Kashima,et al.  Marginalized Kernels Between Labeled Graphs , 2003, ICML.

[43]  Nello Cristianini,et al.  Classification using String Kernels , 2000 .

[44]  Eleazar Eskin,et al.  The Spectrum Kernel: A String Kernel for SVM Protein Classification , 2001, Pacific Symposium on Biocomputing.

[45]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[46]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[47]  Thomas Gärtner,et al.  On Graph Kernels: Hardness Results and Efficient Alternatives , 2003, COLT.