Unifying Logical and Statistical AI

Intelligent agents must be able to handle the complexity and uncertainty of the real world. Logical AI has focused mainly on the former, and statistical AI on the latter. Markov logic combines the two by attaching weights to first-order formulas and viewing them as templates for features of Markov networks. Inference algorithms for Markov logic draw on ideas from satisfiability, Markov chain Monte Carlo and knowledge-based model construction. Learning algorithms are based on the voted perceptron, pseudo-likelihood and inductive logic programming. Markov logic has been successfully applied to a wide variety of problems in natural language understanding, vision, computational biology, social networks and others, and is the basis of the open-source Alchemy system.

[1]  Dejing Dou,et al.  Learning to Refine an Automatically Extracted Knowledge Base Using Markov Logic , 2012, 2012 IEEE 12th International Conference on Data Mining.

[2]  P. Damlen,et al.  Gibbs sampling for Bayesian non‐conjugate and hierarchical models by using auxiliary variables , 1999 .

[3]  J. R. Quinlan.  Learning Logical Definitions from Relations , 1990 .

[4]  Tuyen N. Huynh,et al.  Exact Lifted Inference with Distinct Soft Evidence on Every Object , 2012, AAAI.

[5]  Michael Collins,et al.  Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms , 2002, EMNLP.

[6]  Nils J. Nilsson,et al.  Probabilistic Logic * , 2022 .

[7]  J. W. Lloyd,et al.  Foundations of logic programming; (2nd extended ed.) , 1987 .

[8]  J. A. Robinson,et al.  A Machine-Oriented Logic Based on the Resolution Principle , 1965, JACM.

[9]  Pedro M. Domingos,et al.  Entity Resolution with Markov Logic , 2006, Sixth International Conference on Data Mining (ICDM'06).

[10]  Pedro M. Domingos,et al.  Learning Markov logic network structure via hypergraph lifting , 2009, ICML '09.

[11]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[12]  Joseph Y. Halpern,et al.  From Statistical Knowledge Bases to Degrees of Belief , 1996, Artif. Intell..

[13]  Robert P. Goldman,et al.  From knowledge bases to decision models , 1992, The Knowledge Engineering Review.

[14]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[15]  Pedro M. Domingos,et al.  Efficient Weight Learning for Markov Logic Networks , 2007, PKDD.

[16]  Pedro M. Domingos,et al.  Memory-Efficient Inference in Relational Domains , 2006, AAAI.

[17]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[18]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[19]  Luc De Raedt,et al.  Towards Combining Inductive Logic Programming with Bayesian Networks , 2001, ILP.

[20]  Joseph Y. Halpern.  An Analysis of First-Order Logics of Probability , 1989, IJCAI.

[21]  Mathias Niepert,et al.  Markov Chains on Orbits of Permutation Groups , 2012, UAI.

[22]  Hoifung Poon,et al.  Unsupervised Semantic Parsing , 2009, EMNLP.

[23]  Pedro M. Domingos,et al.  Statistical predicate invention , 2007, ICML '07.

[24]  Sebastian Riedel.  Improving the Accuracy and Efficiency of MAP Inference for Markov Logic , 2008, UAI.

[25]  Pedro M. Domingos,et al.  Lifted First-Order Belief Propagation , 2008, AAAI.

[26]  Heiner Stuckenschmidt,et al.  A Probabilistic-Logical Framework for Ontology Matching , 2010, AAAI.

[27]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[28]  Ewan Klein,et al.  Genic interaction extraction with semantic and syntactic chains , 2005 .

[29]  Pedro M. Domingos,et al.  Joint Inference in Information Extraction , 2007, AAAI.

[30]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[31]  Dan Roth,et al.  On the Hardness of Approximate Reasoning , 1993, IJCAI.

[32]  Stanley Wasserman,et al.  Social Network Analysis: Methods and Applications , 1994, Structural analysis in the social sciences.

[33]  Matthew Richardson,et al.  Mining knowledge-sharing sites for viral marketing , 2002, KDD.

[34]  Luc De Raedt,et al.  Clausal Discovery , 1997, Machine Learning.

[35]  Fahiem Bacchus,et al.  Representing and reasoning with probabilistic knowledge , 1988 .

[36]  Judea Pearl,et al.  Chapter 2 – BAYESIAN INFERENCE , 1988 .

[37]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[38]  Dan Roth,et al.  Lifted First-Order Probabilistic Inference , 2005, IJCAI.

[39]  Pedro M. Domingos,et al.  A General Method for Reducing the Complexity of Relational Inference and its Application to MCMC , 2008, AAAI.

[40]  Pedro M. Domingos,et al.  Extracting Semantic Networks from Text Via Relational Clustering , 2008, ECML/PKDD.

[41]  Martin Fodslette Meiller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1993 .

[42]  Pedro M. Domingos,et al.  Learning the structure of Markov logic networks , 2005, ICML.

[43]  Pedro M. Domingos,et al.  Hypergraph Lifting for Structure Learning in Markov Logic Networks , 2009 .

[44]  Pedro M. Domingos,et al.  Sound and Efficient Inference with Probabilistic and Deterministic Dependencies , 2006, AAAI.

[45]  Matthew Richardson,et al.  Mining the network value of customers , 2001, KDD '01.

[46]  Fahiem Bacchus,et al.  Representing and reasoning with probabilistic knowledge - a logical approach to probabilities , 1991 .

[47]  Pedro M. Domingos,et al.  Discriminative Training of Markov Logic Networks , 2005, AAAI.

[48]  J. Besag.  Statistical Analysis of Non-Lattice Data , 1975 .

[49]  Pedro M. Domingos,et al.  Markov Logic: An Interface Layer for Artificial Intelligence , 2009, Markov Logic: An Interface Layer for Artificial Intelligence.

[50]  Ofer Meshi,et al.  Template Based Inference in Symmetric Relational Markov Random Fields , 2007, UAI.

[51]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[52]  Pedro M. Domingos,et al.  A Language for Relational Decision Theory , 2009 .

[53]  Bart Selman,et al.  Towards Efficient Sampling: Exploiting Random Walk Strategies , 2004, AAAI.

[54]  Guy Van den Broeck,et al.  Conditioning in First-Order Knowledge Compilation and Lifted Probabilistic Inference , 2012, AAAI.

[55]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[56]  Hung Hai Bui,et al.  Lifted Tree-Reweighted Variational Inference , 2014, UAI.

[57]  Jennifer Neville,et al.  Dependency networks for relational data , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[58]  W. Freeman,et al.  Generalized Belief Propagation , 2000, NIPS.

[59]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  Guy Van den Broeck.  On the Complexity and Approximation of Binary Evidence in Lifted Inference , 2013, StarAI@AAAI.

[61]  Pedro M. Domingos,et al.  Approximate Lifting Techniques for Belief Propagation , 2014, AAAI.

[62]  Michael R. Genesereth,et al.  Logical foundations of artificial intelligence , 1987 .

[63]  Matthew Richardson,et al.  The Alchemy System for Statistical Relational AI: User Manual , 2007 .

[64]  S. Muggleton.  Stochastic Logic Programs , 1996 .

[65]  Raymond J. Mooney,et al.  Bottom-up learning of Markov logic network structure , 2007, ICML '07.

[66]  Bart Selman,et al.  A general stochastic approach to solving problems with hard and soft constraints , 1996, Satisfiability Problem: Theory and Applications.

[67]  John Wylie Lloyd,et al.  Foundations of Logic Programming , 1987, Symbolic Computation.

[68]  Pedro M. Domingos,et al.  Markov Logic in Infinite Domains , 2007, UAI.

[69]  Mark Craven,et al.  Relational Learning with Statistical Predicate Invention: Better Models for Hypertext , 2001, Machine Learning.