Probabilistic modelling, inference and learning using logical theories

This paper provides a study of probabilistic modelling, inference and learning in a logic-based setting. We show how probability densities, being functions, can be represented and reasoned with naturally and directly in higher-order logic, an expressive formalism not unlike the (informal) everyday language of mathematics. We give efficient inference algorithms and illustrate the general approach with a diverse collection of applications. Some learning issues are also considered.

[1]  Leon Henkin,et al.  Completeness in the theory of types , 1950, Journal of Symbolic Logic.

[2]  John W. Lloyd,et al.  Knowledge Representation, Computation, and Learning in Higher-order Logic , 2002 .

[3]  Stuart J. Russell,et al.  BLOG: Probabilistic Models with Unknown Objects , 2005, IJCAI.

[4]  John W. Lloyd,et al.  Modal Functional Logic Programming , 2007 .

[5]  Ronald Fagin,et al.  Reasoning about knowledge and probability , 1988, JACM.

[6]  Taisuke Sato,et al.  PRISM: A Language for Symbolic-Statistical Modeling , 1997, IJCAI.

[7]  Joseph Y. Halpern.  Reasoning about uncertainty , 2003 .

[8]  J. W. Lloyd.  Logic and Learning , 2003 .

[9]  Johann Eder,et al.  Logic and Databases , 1992, Advanced Topics in Artificial Intelligence.

[10]  Laks V. S. Lakshmanan,et al.  Modeling Uncertainty in Deductive Databases , 1994, DEXA.

[11]  George Boole,et al.  Studies in logic and probability , 1952 .

[12]  John W. Lloyd,et al.  Programming in an Integrated Functional and Logic Language , 1999, J. Funct. Log. Program..

[13]  David Poole,et al.  First-order probabilistic inference , 2003, IJCAI.

[14]  Avi Pfeffer,et al.  The Design and Implementation of IBAL: A General-Purpose Probabilistic Language , 2005 .

[15]  Peter A. Flach,et al.  Combining Bayesian Networks with Higher-Order Data Representations , 2005, Probabilistic, Logical and Relational Learning.

[16]  J. Nelson Rushton,et al.  Probabilistic reasoning with answer sets , 2004, Theory and Practice of Logic Programming.

[17]  Nevin L. Zhang,et al.  A simple approach to Bayesian network computations , 1994 .

[18]  John W. Lloyd,et al.  Probabilistic reasoning in a classical logic , 2009, J. Appl. Log..

[19]  Luc De Raedt,et al.  Probabilistic logic learning , 2003, SKDD.

[20]  Keith L. Clark,et al.  Negation as Failure , 1987, Logic and Data Bases.

[21]  Alonzo Church,et al.  A formulation of the simple theory of types , 1940, Journal of Symbolic Logic.

[22]  G. Boole.  An Investigation of the Laws of Thought: On which are founded the mathematical theories of logic and probabilities , 2007 .

[23]  Alex M. Andrew,et al.  Logic for Learning: Learning Comprehensible Theories from Structured Data , 2004 .

[24]  Eyal Amir,et al.  Probabilistic Modal Logic , 2007, AAAI.

[25]  M. Erwig,et al.  Probabilistic Functional Programming in Haskell , 2005 .

[26]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[27]  Daniel Leivant,et al.  Higher order logic , 1994, Handbook of Logic in Artificial Intelligence and Logic Programming.

[28]  De Raedt,et al.  Advances in Inductive Logic Programming , 1996 .

[29]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[30]  Simon Thompson,et al.  Type theory and functional programming , 1991, International computer science series.

[31]  Dov M. Gabbay,et al.  Handbook of logic in artificial intelligence and logic programming (vol. 1) , 1993 .

[32]  J. W. Lloyd,et al.  Logic for Learning , 2003, Cognitive Technologies.

[33]  Michael Gelfond,et al.  Classical negation in logic programs and disjunctive databases , 1991, New Generation Computing.

[34]  John W. Lloyd,et al.  Personalisation for user agents , 2005, AAMAS '05.

[35]  V. S. Subrahmanian,et al.  Hybrid Probabilistic Programs , 2000, J. Log. Program..

[36]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[37]  John W. Lloyd,et al.  Learning Modal Theories , 2007, ILP.

[38]  Alan Robinson,et al.  Handbook of automated reasoning , 2001 .

[39]  S. Muggleton.  Stochastic Logic Programs , 1996 .

[40]  Gilles Dowek,et al.  Higher-Order Unification and Matching , 2001, Handbook of Automated Reasoning.

[41]  Nils J. Nilsson,et al.  Probabilistic Logic * , 2022 .

[42]  David Poole,et al.  Logic, Knowledge Representation, and Bayesian Decision Theory , 2000, Computational Logic.

[43]  Martin Erwig,et al.  Functional Pearls: Probabilistic functional programming in Haskell , 2006, J. Funct. Program..

[44]  Zakaria Maamar,et al.  Proceedings of the Workshop on Service-Oriented Computing and Agent Based Engineering (SOCABE 2005) held in conjunction with the 4th International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2005), Utrecht, Netherlands, 25-29 July 2005 , 2005 .

[45]  Jon Williamson,et al.  Part I Frameworks for Probability § 2 Variables , 2008 .

[46]  Norman Ramsey,et al.  Stochastic lambda calculus and monads of probability distributions , 2002, POPL '02.

[47]  Sebastian Thrun,et al.  A probabilistic language based upon sampling functions , 2005, POPL '05.

[48]  X. Jin.  Factor graphs and the Sum-Product Algorithm , 2002 .

[49]  Luc De Raedt,et al.  Bayesian Logic Programming: Theory and Tool , 2007 .

[50]  Joseph Y. Halpern.  An Analysis of First-Order Logics of Probability , 1989, IJCAI.

[51]  John W. Lloyd,et al.  Learning Comprehensible Theories from Structured Data , 2002, Machine Learning Summer School.

[52]  R. J. Fynmore.  F. M. N , 1908 .

[53]  L. Goble.  The Blackwell guide to philosophical logic , 2001 .

[54]  Herbert B. Enderton,et al.  A mathematical introduction to logic , 1972 .

[55]  V. S. Subrahmanian,et al.  Probabilistic Logic Programming , 1992, Inf. Comput..

[56]  Jörg Flum,et al.  Mathematical logic , 1985, Undergraduate texts in mathematics.

[57]  Peter B. Andrews.  An introduction to mathematical logic and type theory - to truth through proof , 1986, Computer science and applied mathematics.

[58]  Stuart J. Russell,et al.  First-Order Probabilistic Languages: Into the Unknown , 2007, ILP.

[59]  John W. Lloyd,et al.  An Architecture for Rational Agents , 2005, DALT.

[60]  William M. Farmer,et al.  The seven virtues of simple type theory , 2008, J. Appl. Log..

[61]  John Wylie Lloyd,et al.  Foundations of Logic Programming , 1987, Symbolic Computation.

[62]  Theodore Hailperin,et al.  Probability logic , 1984, Notre Dame J. Formal Log..

[63]  Lloyd Allison,et al.  Models for machine learning and data mining in functional programming , 2004, Journal of Functional Programming.

[64]  Dan Roth,et al.  Lifted First-Order Probabilistic Inference , 2005, IJCAI.