Lazy Factored Inference for Functional Probabilistic Programming

Probabilistic programming provides the means to represent and reason about complex probabilistic models using programming language constructs. Even simple probabilistic programs can produce models with infinitely many variables. Factored inference algorithms are widely used for probabilistic graphical models, but cannot be applied to these programs because all the variables and factors have to be enumerated. In this paper, we present a new inference framework, lazy factored inference (LFI), that enables factored algorithms to be used for models with infinitely many variables. LFI expands the model to a bounded depth and uses the structure of the program to precisely quantify the effect of the unexpanded part of the model, producing lower and upper bounds to the probability of the query.

[1]  Thomas Hofmann,et al.  TrueSkill™: A Bayesian Skill Rating System , 2007 .

[2]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[3]  Luc De Raedt,et al.  On the implementation of the probabilistic logic programming language ProbLog , 2010, Theory and Practice of Logic Programming.

[4]  Avi Pfeffer,et al.  Semantics and Inference for Recursive Probability Models , 2000, AAAI/IAAI.

[5]  Chung-chieh Shan,et al.  Embedded Probabilistic Programming , 2009, DSL.

[6]  David Poole,et al.  Logic programming, abduction and probability , 1993, New Generation Computing.

[7]  Peter Haddawy,et al.  Generating Bayesian Networks from Probablity Logic Knowledge Bases , 1994, UAI.

[8]  Robert P. Goldman,et al.  Dynamic construction of belief networks , 1990, UAI.

[9]  Tom Minka,et al.  TrueSkillTM: A Bayesian Skill Rating System , 2006, NIPS.

[10]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[11]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[12]  Stuart J. Russell,et al.  BLOG: Probabilistic Models with Unknown Objects , 2005, IJCAI.

[13]  Jung-Fu Cheng,et al.  Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm , 1998, IEEE J. Sel. Areas Commun..

[14]  Avi Pfeffer,et al.  The Design and Implementation of IBAL: A General-Purpose Probabilistic Language , 2005 .

[15]  David A. McAllester,et al.  Effective Bayesian Inference for Stochastic Programs , 1997, AAAI/IAAI.

[16]  C. Q. Lee,et al.  The Computer Journal , 1958, Nature.

[17]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[18]  Alexander T. Ihler,et al.  Accuracy Bounds for Belief Propagation , 2007, UAI.

[19]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[20]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[21]  Guy Van den Broeck,et al.  k-Optimal: A Novel Approximate Inference Algorithm for ProbLog , 2011, ILP.

[22]  Nevin L. Zhang,et al.  A simple approach to Bayesian network computations , 1994 .

[23]  Rina Dechter,et al.  Bucket Elimination: A Unifying Framework for Reasoning , 1999, Artif. Intell..

[24]  John Hughes,et al.  Why Functional Programming Matters , 1989, Comput. J..

[25]  Ross D. Shachter.  Probabilistic Inference and Influence Diagrams , 1988, Oper. Res..

[26]  Luc De Raedt,et al.  Explanation-Based Approximate Weighted Model Counting for Probabilistic Logics , 2014, AAAI.

[27]  Hilbert J. Kappen,et al.  Bounds on marginal probability distributions , 2008, NIPS.

[28]  Anders L. Madsen,et al.  An Empirical Evaluation of Possible Variations of Lazy Propagation , 2004, UAI.

[29]  Terrance E. Boult,et al.  Pruning bayesian networks for efficient computation , 1990, UAI.

[30]  Serafín Moral,et al.  Lazy evaluation in penniless propagation over join trees , 2002, Networks.