A Survey of First-Order Probabilistic Models

There has been a long standing division in Artificial Intelligence between logical and probabilistic reasoning approaches. While probabilistic models can deal well with inherent uncertainty in many real-world domains, they operate on a mostly propositional level. Logic systems, on the other hand, can deal with much richer representations, especially first-order ones, but treat uncertainty only in limited ways. Therefore, an integration of these types of inference is highly desirable, and many approaches have been proposed, especially from the 1990s on. These solutions come from many different subfields and vary greatly in language, features and (when available at all) inference algorithms. Therefore their relation to each other is not always clear, as well as their semantics. In this survey, we present the main aspects of the solutions proposed and group them according to language, semantics and inference algorithm. In doing so, we draw relations between them and discuss particularly important choices and tradeoffs.

[1]  Nevin L. Zhang,et al.  A simple approach to Bayesian network computations , 1994 .

[2]  Walter R. Gilks,et al.  BUGS - Bayesian inference Using Gibbs Sampling Version 0.50 , 1995 .

[3]  Robert P. Goldman,et al.  A Language for Construction of Belief Networks , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Edward H. Shortliffe,et al.  A rule-based computer program for advising physicians regarding antimicrobial therapy selection , 1974, ACM '74.

[5]  Avi Pfeffer,et al.  Probabilistic Frame-Based Systems , 1998, AAAI/IAAI.

[6]  Peter A. Flach,et al.  1BC2: A True First-Order Bayesian Classifier , 2002, ILP.

[7]  David Maxwell Chickering,et al.  Dependency Networks for Inference, Collaborative Filtering, and Data Visualization , 2000, J. Mach. Learn. Res..

[8]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[9]  Laks V. S. Lakshmanan,et al.  An Epistemic Foundation for Logic Programming with Uncertainty , 1994, FSTTCS.

[10]  David Poole,et al.  Probabilistic Horn Abduction and Bayesian Networks , 1993, Artif. Intell..

[11]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[12]  Luc De Raedt,et al.  Bayesian Logic Programs , 2001, ILP Work-in-progress reports.

[13]  Wray L. Buntine.  Operations for Learning with Graphical Models , 1994, J. Artif. Intell. Res..

[14]  Beat Wüthrich.  Probabilistic Knowledge Bases , 1995, IEEE Trans. Knowl. Data Eng..

[15]  Kathryn B. Laskey.  First-order Bayesian logic , 2005 .

[16]  Laks V. S. Lakshmanan,et al.  Probabilistic Deductive Databases , 1994, ILPS.

[17]  Joseph Y. Halpern.  An Analysis of First-Order Logics of Probability , 1989, IJCAI.

[18]  Thomas Lukasiewicz,et al.  Probabilistic Deduction with Conditional Constraints over Basic Events , 2011, KR.

[19]  H. Gaifman.  Concerning measures in first order calculi , 1964 .

[20]  Nils J. Nilsson,et al.  Probabilistic Logic * , 2022 .

[21]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[22]  Stefan Riezler,et al.  Probabilistic Constraint Logic Programming , 1997, ArXiv.

[23]  James Cussens,et al.  Loglinear models for first-order probabilistic reasoning , 1999, UAI.

[24]  Gai CarSO.  A Logic for Reasoning about Probabilities * , 2004 .

[25]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[26]  Luc De Raedt,et al.  Towards Combining Inductive Logic Programming with Bayesian Networks , 2001, ILP.

[27]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[28]  Hector J. Levesque,et al.  Expressiveness and tractability in knowledge representation and reasoning 1 , 1987, Comput. Intell..

[29]  Fahiem Bacchus,et al.  Representing and reasoning with probabilistic knowledge - a logical approach to probabilities , 1991 .

[30]  Marvin Minsky,et al.  A framework for representing knowledge , 1974 .

[31]  R F Harris.  Smoking causes cancer! , 1997, Current Biology.

[32]  J. R. Quinlan.  Learning Logical Definitions from Relations , 1990 .

[33]  Paolo Frasconi,et al.  Hidden Tree Markov Models for Document Image Classification , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[35]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[36]  David Poole,et al.  First-order probabilistic inference , 2003, IJCAI.

[37]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[38]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Manfred Jaeger,et al.  Relational Bayesian Networks , 1997, UAI.

[40]  Luc De Raedt,et al.  Probabilistic inductive logic programming , 2004 .

[41]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[42]  Dan Roth,et al.  Lifted First-Order Probabilistic Inference , 2005, IJCAI.

[43]  Lise Getoor,et al.  Learning Probabilistic Relational Models with Structural Uncertainty , 2000 .

[44]  Marc Snir,et al.  Probabilities over rich languages, testing and randomness , 1982, Journal of Symbolic Logic.

[45]  Steve Young,et al.  Applications of stochastic context-free grammars using the Inside-Outside algorithm , 1990 .

[46]  Ehud Y. Shapiro,et al.  Logic Programs With Uncertainties: A Tool for Implementing Rule-Based Systems , 1983, IJCAI.

[47]  J. Baker.  Trainable grammars for speech recognition , 1979 .

[48]  Alex Waibel,et al.  Readings in speech recognition , 1990 .

[49]  Taisuke Sato,et al.  A Viterbi-like algorithm and EM learning for statistical abduction , 2000 .

[50]  Michael Kifer,et al.  On the Semantics of Rule-Based Expert Systems with Uncertainty , 1988, ICDT.

[51]  Joxan Jaffar,et al.  Constraint logic programming , 1987, POPL '87.

[52]  Stuart J. Russell,et al.  BLOG: Probabilistic Models with Unknown Objects , 2005, IJCAI.

[53]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[54]  Andrew P. Sage,et al.  Uncertainty in Artificial Intelligence , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[55]  Peter Haddawy,et al.  Anytime Deduction for Probabilistic Logic , 1994, Artif. Intell..

[56]  Joseph Y. Halpern,et al.  Irrelevance and Conditioning in First-Order Probabilistic Logic , 1996, AAAI/IAAI, Vol. 1.

[57]  Peter J. F. Lucas,et al.  Certainty-factor-like structures in Bayesian belief networks , 2001, Knowl. Based Syst..

[58]  Alon Y. Halevy,et al.  P-CLASSIC: A Tractable Probablistic Description Logic , 1997, AAAI/IAAI.

[59]  Jennifer Neville,et al.  Statistical models and analysis techniques for learning in relational data , 2006 .

[60]  S. Muggleton.  Stochastic Logic Programs , 1996 .

[61]  Luc De Raedt,et al.  Clausal Discovery , 1997, Machine Learning.

[62]  Avi Pfeffer,et al.  Learning Probabilities for Noisy First-Order Rules , 1997, IJCAI.

[63]  Pedro M. Domingos,et al.  Relational Markov models and their application to adaptive web navigation , 2002, KDD.

[64]  Jennifer Neville,et al.  Simple estimators for relational Bayesian classifiers , 2003, Third IEEE International Conference on Data Mining.

[65]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[66]  Peter A. Flach,et al.  IBC: A First-Order Bayesian Classifier , 1999, ILP.

[67]  J. Besag.  Statistical Analysis of Non-Lattice Data , 1975 .

[68]  Edward H. Shortliffe,et al.  Rule Based Expert Systems: The Mycin Experiments of the Stanford Heuristic Programming Project (The Addison-Wesley series in artificial intelligence) , 1984 .

[69]  Robert P. Goldman,et al.  From knowledge bases to decision models , 1992, The Knowledge Engineering Review.

[70]  Avi Pfeffer,et al.  Semantics and Inference for Recursive Probability Models , 2000, AAAI/IAAI.

[71]  Sabine Glesner,et al.  Constructing Flexible Dynamic Belief Networks from First-Order Probalistic Knowledge Bases , 1995, ECSQARU.

[72]  Alex S. Taylor,et al.  Machine intelligence , 2009, CHI.

[73]  J. Nelson Rushton,et al.  Probabilistic reasoning with answer sets , 2004, Theory and Practice of Logic Programming.

[74]  John S. Breese,et al.  CONSTRUCTION OF BELIEF AND DECISION NETWORKS , 1992, Comput. Intell..

[75]  Dan Roth,et al.  Relational Representations that Facilitate Learning , 1999, KR.

[76]  Peter Haddawy,et al.  Probabilistic Logic Programming and Bayesian Networks , 1995, ASIAN.

[77]  David Poole,et al.  A Dynamic Approach to Probabilistic Inference using Bayesian Networks , 1990, UAI 1990.

[78]  Nicos Angelopoulos.  Probabilistic Finite Domains: A Brief Overview , 2002, ICLP.

[79]  David Heckerman,et al.  Probabilistic Interpretation for MYCIN's Certainty Factors , 1990, UAI.

[80]  David Poole,et al.  The Independent Choice Logic for Modelling Multiple Agents Under Uncertainty , 1997, Artif. Intell..

[81]  Peter Haddawy,et al.  Generating Bayesian Networks from Probablity Logic Knowledge Bases , 1994, UAI.

[82]  Taisuke Sato,et al.  PRISM: A Language for Symbolic-Statistical Modeling , 1997, IJCAI.

[83]  H. Kyburg,et al.  Logical foundations of probability , 1951 .

[84]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.

[85]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[86]  James Cussens,et al.  CLP(BN): Constraint Logic Programming for Probabilistic Knowledge , 2002, Probabilistic Inductive Logic Programming.

[87]  Eric Mjolsness,et al.  Labeled graph notations for graphical models Extended Report , 2004 .

[88]  V. S. Subrahmanian,et al.  Probabilistic Logic Programming , 1992, Inf. Comput..

[89]  Maurice Bruynooghe,et al.  Logical Bayesian Networks and Their Relation to Other Probabilistic Logical Models , 2005, BNAIC.