Multidimensional k-Interaction Classifier: Taking Advantage of All the Information Contained in Low Order Interactions

This work presents a multidimensional classifier described in terms of interaction factors called multidimensional k-interaction classifier. The classifier is based on a probabilistic model composed of the product of all the interaction factors of order lower or equal to k and it takes advantage of all the information contained in them. The proposed classifier does not require a model selection step and its complexity is controlled by the regularization parameter k. Multidimensional k-interaction classifier is a generalization of the Kikuchi-Bayes classifier (Jakulin et al. 2004) to the multidimensional classification problem. The proposed multidimensional classifier is especially appropriate for small k values and for low dimensional domains. Multidimensional k-interaction classifier has shown a competitive behavior in difficult artificial domains for which the low order marginal distributions are almost uniform.

[1]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[2]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[3]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[4]  Concha Bielza,et al.  Multi-dimensional classification with Bayesian networks , 2011, Int. J. Approx. Reason..

[5]  Concha Bielza,et al.  Bayesian Chain Classifiers for Multidimensional Classification , 2011, IJCAI.

[6]  Linda C. van der Gaag,et al.  Multi-dimensional Bayesian Network Classifiers , 2006, Probabilistic Graphical Models.

[7]  Tony Jebara,et al.  Machine Learning: Discriminative and Generative , 2012 .

[8]  I. Bratko,et al.  Kikuchi-Bayes: Factorized Models for Approximate Classification in Closed Form , 2004 .

[9]  Jack J. Dongarra,et al.  Matrix Eigensystem Routines — EISPACK Guide Extension , 1977, Lecture Notes in Computer Science.

[10]  W. J. McGill.  Multivariate information transmission , 1954, Trans. IRE Prof. Group Inf. Theory.

[11]  José Antonio Lozano,et al.  A general framework for the statistical analysis of the sources of variance for classification error estimators , 2013, Pattern Recognit..

[12]  J. Kirkwood.  Statistical Mechanics of Fluid Mixtures , 1935 .

[13]  Luis Enrique Sucar,et al.  A Two-Step Method to Learn Multidimensional Bayesian Network Classifiers Based on Mutual Information Measures , 2011, FLAIRS.

[14]  Linda C. van der Gaag,et al.  Inference and Learning in Multi-dimensional Bayesian Network Classifiers , 2007, ECSQARU.

[15]  Klaus Krippendorff,et al.  Information Theory: Structural Models for Qualitative Data. , 1988 .

[16]  Michael Clarke,et al.  Symbolic and Quantitative Approaches to Reasoning and Uncertainty , 1991, Lecture Notes in Computer Science.

[17]  Aleks Jakulin,et al.  Attribute Interactions in Machine Learning , 2003 .

[18]  R. Kikuchi.  A Theory of Cooperative Phenomena , 1951 .

[19]  Gavin Brown,et al.  Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection , 2012, J. Mach. Learn. Res..