Stochastic lambda calculus and monads of probability distributions

Probability distributions are useful for expressing the meanings of probabilistic languages, which support formal modeling of and reasoning about uncertainty. Probability distributions form a monad, and the monadic definition leads to a simple, natural semantics for a stochastic lambda calculus, as well as simple, clean implementations of common queries. But the monadic implementation of the expectation query can be much less efficient than current best practices in probabilistic modeling. We therefore present a language of measure terms, which can not only denote discrete probability distributions but can also support the best known modeling techniques. We give a translation of stochastic lambda calculus into measure terms. Whether one translates into the probability monad or into measure terms, the results of the translations denote the same probability distribution.

[1]  Avi Pfeffer,et al.  Semantics and Inference for Recursive Probability Models , 2000, AAAI/IAAI.

[2]  N. Saheb-Djahromi,et al.  Probabilistic LCF , 1978, International Symposium on Mathematical Foundations of Computer Science.

[3]  Stefan Arnborg,et al.  Efficient algorithms for combinatorial problems on graphs with bounded decomposability — A survey , 1985, BIT.

[4]  Michael I. Jordan,et al.  Variational Probabilistic Inference and the QMR-DT Network , 2011, J. Artif. Intell. Res..

[5]  Nevin L. Zhang,et al.  A simple approach to Bayesian network computations , 1994 .

[6]  John Hughes,et al.  The Design of a Pretty-printing Library , 1995, Advanced Functional Programming.

[7]  Olivier Danvy Type-Directed Partial Evaluation , 1998, Partial Evaluation.

[8]  Judea Pearl,et al.  Chapter 2 – BAYESIAN INFERENCE , 1988 .

[9]  S. Muggleton Stochastic Logic Programs , 1996 .

[10]  W. Rudin Real and complex analysis , 1968 .

[11]  Norman Ramsey,et al.  Literate programming simplified , 1994, IEEE Software.

[12]  Michael B. Smyth,et al.  Power Domains and Predicate Transformers: A Topological View , 1983, ICALP.

[13]  Albert Benveniste,et al.  A Calculus of Stochastic Systems for the Specification, Simulation, and Hidden State Estimation of Mixed Stochastic/Nonstochastic Systems , 1994, Theor. Comput. Sci..

[14]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[15]  Claire Jones,et al.  Probabilistic non-determinism , 1990 .

[16]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP.

[17]  Rina Dechter,et al.  Bucket elimination: A unifying framework for probabilistic inference , 1996, UAI.

[18]  Kathryn B. Laskey,et al.  Constructing Situation Specific Belief Networks , 1998, UAI.

[19]  Philip Wadler,et al.  The essence of functional programming , 1992, POPL '92.

[20]  David A. McAllester,et al.  Effective Bayesian Inference for Stochastic Programs , 1997, AAAI/IAAI.

[21]  W. Rudin Real and complex analysis, 3rd ed. , 1987 .

[22]  Zhaoyu Li,et al.  Efficient inference in Bayes networks as a combinatorial optimization problem , 1994, Int. J. Approx. Reason..

[23]  Avi Pfeffer,et al.  IBAL: A Probabilistic Rational Programming Language , 2001, IJCAI.

[24]  Philip Wadler,et al.  The essence of functional programming (Invited talk) , 1997 .

[25]  ArnborgStefan Efficient algorithms for combinatorial problems on graphs with bounded, decomposabilitya survey , 1985 .

[26]  John Hughes,et al.  Why Functional Programming Matters , 1989, Comput. J..

[27]  C. Jones,et al.  A probabilistic powerdomain of evaluations , 1989, [1989] Proceedings. Fourth Annual Symposium on Logic in Computer Science.

[28]  N. Saheb-Djahromi,et al.  CPO'S of Measures for Nondeterminism , 1980, Theor. Comput. Sci..

[29]  Andrew W. Appel,et al.  Compiling with Continuations , 1991 .

[30]  Michael I. Jordan Learning in Graphical Models , 1999, NATO ASI Series.

[31]  Ian H. Witten,et al.  Arithmetic coding for data compression , 1987, CACM.

[32]  Radha Jagadeesan,et al.  Stochastic processes as concurrent constraint programs , 1999, POPL '99.

[33]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[34]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .