New Advances in Logic-Based Probabilistic Modeling by PRISM

We review a logic-based modeling language PRISM and report recent developments including belief propagation by the generalized inside-outside algorithm and generative modeling with constraints. The former implies PRISM subsumes belief propagation at the algorithmic level. We also compare the performance of PRISM with state-of-theart systems in statistical natural language processing and probabilistic inference in Bayesian networks respectively, and show that PRISM is reasonably competitive.

[1]  V. S. Subrahmanian,et al.  Hybrid Probabilistic Programs , 2000, J. Log. Program..

[2]  Luís Moniz Pereira,et al.  Computational Logic — CL 2000 , 2000, Lecture Notes in Computer Science.

[3]  Noah A. Smith,et al.  Compiling Comp Ling: Weighted Dynamic Programming and the Dyna Language , 2005, HLT.

[4]  Adnan Darwiche,et al.  A compiler for deterministic, decomposable negation normal form , 2002, AAAI/IAAI.

[5]  Maurice Bruynooghe,et al.  Logic programs with annotated disjunctions , 2004, NMR.

[6]  Manfred Jaeger,et al.  Compiling relational Bayesian networks for exact inference , 2006, Int. J. Approx. Reason..

[7]  Yoshitaka Kameya,et al.  Parameter Learning of Logic Programs for Symbolic-Statistical Modeling , 2001, J. Artif. Intell. Res..

[8]  Steven P. Abney.  Stochastic Attribute-Value Grammars , 1996, CL.

[9]  José Manuel Gutiérrez,et al.  Expert Systems and Probabiistic Network Models , 1996 .

[10]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[11]  Neng-Fa Zhou,et al.  Efficient fixpoint computation in linear tabling , 2003, PPDP '03.

[12]  Krzysztof R. Apt,et al.  Logic Programming , 1990, Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics.

[13]  Zhiyi Chi,et al.  Estimation of Probabilistic Context-Free Grammars , 1998, Comput. Linguistics.

[14]  Michael I. Jordan,et al.  Probabilistic Independence Networks for Hidden Markov Probability Models , 1997, Neural Computation.

[15]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[16]  Taisuke Sato,et al.  Efficient EM Learning with Tabulation for Parameterized Logic Programs , 2000, Computational Logic.

[17]  J. Baker.  Trainable grammars for speech recognition , 1979 .

[18]  De Raedt,et al.  Advances in Inductive Logic Programming , 1996 .

[19]  Yusuke Izumi.  Parallel EM Learning for Symbolic-Statistical Models , 2006 .

[20]  L. De Raedt,et al.  Logical Hidden Markov Models , 2011, J. Artif. Intell. Res..

[21]  Ben Taskar,et al.  Learning Probabilistic Models of Relational Structure , 2001, ICML.

[22]  Prakash P. Shenoy,et al.  Axioms for probability and belief-function proagation , 1990, UAI.

[23]  Ivan A. Sag,et al.  Syntactic Theory: A Formal Introduction , 1999, Computational Linguistics.

[24]  Manfred Jaeger,et al.  Complex Probabilistic Modeling with Recursive Relational Bayesian Networks , 2001, Annals of Mathematics and Artificial Intelligence.

[25]  Noah A. Smith,et al.  Compiling Comp Ling: Weighted Dynamic Programming and the Dyna Language , 2005, HLT.

[26]  Rina Dechter,et al.  The Relationship Between AND / OR Search Spaces and Variable Elimination , 2005 .

[27]  Kiyoaki Shirai,et al.  Fast Em Learning of a Family of Pcfgs Kameya, Yoshitaka (ns Solutions) , 2022 .

[28]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[29]  Enrico Pontelli,et al.  Towards a More Practical Hybrid Probabilistic Logic Programming Framework , 2005, PADL.

[30]  David Poole,et al.  The Independent Choice Logic for Modelling Multiple Agents Under Uncertainty , 1997, Artif. Intell..

[31]  Prakash P. Shenoy,et al.  Probability propagation , 1990, Annals of Mathematics and Artificial Intelligence.

[32]  Taisuke Sato,et al.  PRISM: A Language for Symbolic-Statistical Modeling , 1997, IJCAI.

[33]  Helmut Schmid.  A Generative Probability Model for Unification-Based Grammars , 2002, COLING.

[34]  Taisuke Sato,et al.  Bayesian classification of a human custom based on stochastic context-free grammar , 2007, Systems and Computers in Japan.

[35]  David Poole,et al.  Probabilistic Horn Abduction and Bayesian Networks , 1993, Artif. Intell..

[36]  Dale Schuurmans,et al.  Exploiting syntactic, semantic and lexical regularities in language modeling via directed Markov random fields , 2004, 2004 International Symposium on Chinese Spoken Language Processing.

[37]  S. Muggleton.  Stochastic Logic Programs , 1996 .

[38]  Peter Haddawy,et al.  Answering Queries from Context-Sensitive Probabilistic Knowledge Bases , 1997, Theor. Comput. Sci..

[39]  Adnan Darwiche,et al.  Compiling Bayesian Networks with Local Structure , 2005, IJCAI.

[40]  C. S. Wetherell,et al.  Probabilistic Languages: A Review and Some Open Questions , 1980, CSUR.

[41]  V. S. Subrahmanian,et al.  Probabilistic Logic Programming , 1992, Inf. Comput..

[42]  Neng-Fa Zhou,et al.  Generative Modeling with Failure in PRISM , 2005, IJCAI.

[43]  Maurice Bruynooghe,et al.  Logical Bayesian Networks and Their Relation to Other Probabilistic Logical Models , 2005, BNAIC.

[44]  Peter Haddawy,et al.  Anytime Deduction for Probabilistic Logic , 1994, Artif. Intell..

[45]  Luc De Raedt,et al.  Basic Principles of Learning Bayesian Logic Programs , 2008, Probabilistic Inductive Logic Programming.

[46]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[47]  Taisuke Sato.  Modeling Scienti c Theories as PRISM Programs , 2007 .

[48]  Luc De Raedt,et al.  ProbLog: A Probabilistic Prolog and its Application in Link Discovery , 2007, IJCAI.

[49]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[50]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[51]  Ehud Shapiro,et al.  Third International Conference on Logic Programming , 1986 .

[52]  Avi Pfeffer,et al.  Learning Probabilities for Noisy First-Order Rules , 1997, IJCAI.

[53]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[54]  Ken Satoh,et al.  Compiling Bayesian Networks by Symbolic Probability Calculation Based on Zero-Suppressed BDDs , 2007, IJCAI.

[55]  Kees Doets,et al.  From logic to logic programming , 1994, Foundations of computing series.

[56]  J. Nelson Rushton,et al.  Probabilistic reasoning with answer sets , 2004, Theory and Practice of Logic Programming.

[57]  John S. Breese,et al.  CONSTRUCTION OF BELIEF AND DECISION NETWORKS , 1992, Comput. Intell..

[58]  Taisuke Sato,et al.  Inside-Outside Probability Computation for Belief Propagation , 2007, IJCAI.

[59]  Luc De Raedt,et al.  Bayesian Logic Programs , 2001, ILP Work-in-progress reports.

[60]  Andreas Stolcke,et al.  An Efficient Probabilistic Context-Free Parsing Algorithm that Computes Prefix Probabilities , 1994, CL.

[61]  Laks V. S. Lakshmanan,et al.  Probabilistic Deductive Databases , 1994, ILPS.

[62]  Thomas Lukasiewicz,et al.  Probabilistic Deduction with Conditional Constraints over Basic Events , 2011, KR.

[63]  Avi Pfeffer,et al.  IBAL: A Probabilistic Rational Programming Language , 2001, IJCAI.

[64]  James Cussens,et al.  Parameter Estimation in Stochastic Logic Programs , 2001, Machine Learning.

[65]  Fabio Gagliardi Cozman,et al.  Bucket-Tree Elimination for Automated Reasoning , 2001 .

[66]  Taisuke Sato,et al.  Bayesian classification of a human custom based on stochastic context-free grammar , 2007 .

[67]  Stuart J. Russell,et al.  BLOG: Probabilistic Models with Unknown Objects , 2005, IJCAI.

[68]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[69]  Gregory Piatetsky-Shapiro,et al.  Advances in Knowledge Discovery and Data Mining , 2004, Lecture Notes in Computer Science.

[70]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[71]  Peter C. Cheeseman,et al.  Bayesian Classification (AutoClass): Theory and Results , 1996, Advances in Knowledge Discovery and Data Mining.

[72]  Stuart J. Russell,et al.  Approximate Inference for Infinite Contingent Bayesian Networks , 2005, AISTATS.

[73]  Hisao Tamaki,et al.  First Order Compiler: A Deterministic Logic Program Synthesis Algorithm , 1989, J. Symb. Comput..

[74]  Michel Loève,et al.  Probability Theory I , 1977 .

[75]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[76]  Taisuke Sato,et al.  A Statistical Learning Method for Logic Programs with Distribution Semantics , 1995, ICLP.

[77]  Johann Eder,et al.  Logic and Databases , 1992, Advanced Topics in Artificial Intelligence.

[78]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[79]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[80]  Robert P. Goldman,et al.  From knowledge bases to decision models , 1992, The Knowledge Engineering Review.

[81]  Rina Dechter,et al.  The Relationship Between AND/OR Search and Variable Elimination , 2005, UAI.

[82]  Lakhmi C. Jain,et al.  Introduction to Bayesian Networks , 2008 .

[83]  Eugene Charniak,et al.  Tree-Bank Grammars , 1996, AAAI/IAAI, Vol. 2.

[84]  Kathryn B. Laskey.  MEBN: A Logic for Open-World Probabilistic Reasoning , 2006 .

[85]  Fernando Pereira,et al.  Case-factor diagrams for structured probabilistic modeling , 2004, J. Comput. Syst. Sci..

[86]  Michael P. Wellman,et al.  Generalized Queries on Probabilistic Context-Free Grammars , 1996, AAAI/IAAI, Vol. 2.

[87]  Nils J. Nilsson,et al.  Probabilistic Logic * , 2022 .

[88]  Christophe Bérenguer,et al.  A practical comparison of methods to assess sum-of-products , 2003, Reliab. Eng. Syst. Saf..

[89]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[90]  Keith L. Clark,et al.  Negation as Failure , 1987, Logic and Data Bases.

[91]  Hisao Tamaki,et al.  OLD Resolution with Tabulation , 1986, ICLP.

[92]  Taisuke Sato,et al.  Negation Elimination for Finite PCFGs , 2004, LOPSTR.