Propositional Kernels

The pervasive presence of artificial intelligence (AI) in our everyday life has nourished the pursuit of explainable AI. Since the dawn of AI, logic has been widely used to express, in a human-friendly fashion, the internal process that led an (intelligent) system to deliver a specific output. In this paper, we take a step forward in this direction by introducing a novel family of kernels, called Propositional kernels, that construct feature spaces that are easy to interpret. Specifically, Propositional Kernel functions compute the similarity between two binary vectors in a feature space composed of logical propositions of a fixed form. The Propositional kernel framework improves upon the recent Boolean kernel framework by providing more expressive kernels. In addition to the theoretical definitions, we also provide an algorithm (and the source code) to efficiently construct any propositional kernel. An extensive empirical evaluation shows the effectiveness of Propositional kernels on several artificial and benchmark categorical data sets.

[1]  Alexandros Kalousis,et al.  Convex formulations of radius-margin based Support Vector Machines , 2013, ICML.

[2]  Ken Sadohara.  On a capacity control using Boolean kernels for the learning of Boolean functions , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[3]  Chih-Jen Lin,et al.  Radius Margin Bounds for Support Vector Machines with the RBF Kernel , 2002, Neural Computation.

[4]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[5]  Rocco A. Servedio,et al.  Efficiency versus Convergence of Boolean Kernels for On-Line Learning Algorithms , 2001, NIPS.

[6]  Mirko Polato,et al.  A Novel Boolean Kernels Family for Categorical Data † , 2018, Entropy.

[7]  Mani B. Srivastava,et al.  How Can I Explain This to You? An Empirical Study of Deep Neural Network Explanation Methods , 2020, NeurIPS.

[8]  Alex Smola,et al.  Kernel methods in machine learning , 2007, math/0701907.

[9]  Filip Karlo Dosilovic,et al.  Explainable artificial intelligence: A survey , 2018, 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[10]  Mirko Polato,et al.  Radius-Margin Ratio Optimization for Dot-Product Boolean Kernel Learning , 2017, ICANN.

[11]  Mirko Polato,et al.  Boolean kernels for rule based interpretation of support vector machines , 2019, Neurocomputing.

[12]  Antonello Rizzi,et al.  (Hyper)graph Kernels over Simplicial Complexes , 2020, Entropy.

[13]  Alexander J. Smola,et al.  Kernel Machines and Boolean Functions , 2001, NIPS.

[14]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[15]  Mirko Polato,et al.  Kernel based collaborative filtering for very large scale top-N item recommendation , 2016, ESANN.

[16]  Ping Wang,et al.  Research on Face Recognition Based on Boolean Kernel SVM , 2008, 2008 Fourth International Conference on Natural Computation.

[17]  Luiz A. Baccalá,et al.  Kernel Methods for Nonlinear Connectivity Detection , 2019, Entropy.

[18]  Fabio Aiolli,et al.  Learning deep kernels in the space of dot product polynomials , 2017, Machine Learning.

[19]  Rocco A. Servedio,et al.  Maximum Margin Algorithms with Boolean Kernels , 2005, J. Mach. Learn. Res..

[20]  Yoshifumi Kusunoki,et al.  Boolean kernels and clustering with pairwise constraints , 2014, 2014 IEEE International Conference on Granular Computing (GrC).

[21]  Andrew P. Bradley,et al.  Rule extraction from support vector machines: A review , 2010, Neurocomputing.

[22]  Ken Sadohara,et al.  Learning of Boolean Functions Using Support Vector Machines , 2001, ALT.

[23]  Kebin Cui,et al.  Short-Term Load Forecasting Based on the BKF-SVM , 2009, 2009 International Conference on Networks Security, Wireless Communications and Trusted Computing.

[24]  Kebin Cui,et al.  Application of Boolean Kernel Function SVM in Face Recognition , 2009, 2009 International Conference on Networks Security, Wireless Communications and Trusted Computing.

[25]  Assaf J. Kfoury,et al.  Mathematical Logic in Computer Science , 2018, ArXiv.

[26]  Mirko Polato,et al.  Boolean kernels for collaborative filtering in top-N item recommendation , 2018, Neurocomputing.

[27]  Shugang Liu,et al.  Applications of Support Vector Machine Based on Boolean Kernel to Spam Filtering , 2009 .

[28]  Mirko Polato.  Definition and learning of logic-based kernels for categorical data, and application to collaborative filtering , 2018 .

[29]  Wenlong Fu,et al.  Intelligent Fault Identification for Rolling Bearings Fusing Average Refined Composite Multiscale Dispersion Entropy-Assisted Feature Extraction and SVM with Multi-Strategy Enhanced Swarm Optimization , 2021, Entropy.

[30]  Mordechai Ben-Ari.  Mathematical logic for computer science (2. ed.) , 2001 .