Multi-class Prediction Using Stochastic Logic Programs

In this paper, we present a probabilistic method of dealing with multi-class classification using Stochastic Logic Programs (SLPs), a Probabilistic Inductive Logic Programming (PILP) framework that integrates probability, logic representation and learning. Multi-class prediction attempts to classify an observed datum or example into its proper classification given that it has been tested to have multiple predictions. We apply an SLP parameter estimation algorithm to a previous study in the protein fold prediction area and a multi-class classification working example, in which logic programs have been learned by Inductive Logic Programming (ILP) and a large number of multiple predictions have been detected. On the basis of several experiments, we demonstrate that PILP approaches (eg. SLPs) have advantages for solving multi-class prediction problems with the help of learned probabilities. In addition, we show that SLPs outperform ILP plus majority class predictor in both predictive accuracy and result interpretability.

[1]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[2]  Yves Deville,et al.  Multi-class protein fold classification using a new ensemble machine learning approach. , 2003, Genome informatics. International Conference on Genome Informatics.

[3]  S. Ishii,et al.  A multi-class predictor based on a probabilistic model: application to gene expression profiling-based diagnosis of thyroid tumors , 2006, BMC Genomics.

[4]  Kristian Kersting,et al.  TildeCRF: Conditional Random Fields for Logical Sequences , 2006, ECML.

[5]  John Moult,et al.  Rigorous performance evaluation in protein structure modelling and implications for computational biology , 2006, Philosophical Transactions of the Royal Society B: Biological Sciences.

[6]  Dan Roth,et al.  A Sequential Model for Multi-Class Classification , 2001, EMNLP.

[7]  L. De Raedt,et al.  Logical Hidden Markov Models , 2011, J. Artif. Intell. Res..

[8]  Dino Pedreschi,et al.  Machine Learning: ECML 2004 , 2004, Lecture Notes in Computer Science.

[9]  Chris H. Q. Ding,et al.  Multi-class protein fold recognition using support vector machines and neural networks , 2001, Bioinform..

[10]  Luc De Raedt,et al.  Kernels on Prolog Proof Trees: Statistical Learning in the ILP Setting , 2006, J. Mach. Learn. Res..

[11]  James Cussens,et al.  Parameter Estimation in Stochastic Logic Programs , 2001, Machine Learning.

[12]  Stephen Muggleton,et al.  Learning Stochastic Logic Programs , 2000, Electron. Trans. Artif. Intell..

[13]  Neil D. Lawrence,et al.  Missing Data in Kernel PCA , 2006, ECML.

[14]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[15]  Luc De Raedt,et al.  Probabilistic, Logical and Relational Learning - Towards a Synthesis, 30. January - 4. February 2005 , 2006, Probabilistic, Logical and Relational Learning.

[16]  Stephen Muggleton,et al.  The Effect of Relational Background Knowledge on Learning of Protein Three-Dimensional Fold Signatures , 2001, Machine Learning.

[17]  Stephen Muggleton,et al.  Learning Structure and Parameters of Stochastic Logic Programs , 2002, ILP.

[18]  Stephen H Muggleton,et al.  The automatic discovery of structural principles describing protein fold space. , 2003, Journal of molecular biology.

[19]  Henrik Boström,et al.  Resolving rule conflicts with double induction , 2004, Intell. Data Anal..

[20]  C. Chothia,et al.  Understanding protein structure: using scop for fold interpretation. , 1996, Methods in enzymology.

[21]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[22]  Stephen Muggleton,et al.  Relational Rule Induction with CProgol4.4: A Tutorial Introduction , 2001 .

[23]  Thomas Gärtner,et al.  Fisher Kernels for Logical Sequences , 2004, ECML.

[24]  Dan Roth,et al.  Constraint Classification: A New Approach to Multiclass Classification , 2002, ALT.

[25]  S. Muggleton Stochastic Logic Programs , 1996 .

[26]  M J Sternberg,et al.  Automated discovery of structural signatures of protein fold and function. , 2001, Journal of molecular biology.