Formally justified and modular Bayesian inference for probabilistic programs

Engineering and Physical Sciences Research Council, Cambridge Trust, Cambridge-Tuebingen programme

[1]  Radford M. Neal.  Annealed importance sampling , 1998, Stat. Comput..

[2]  C. Andrieu,et al.  The pseudo-marginal approach for efficient Monte Carlo computations , 2009, 0903.5480.

[3]  Norman Ramsey,et al.  Stochastic lambda calculus and monads of probability distributions , 2002, POPL '02.

[4]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[5]  Maciej Piróg,et al.  Eilenberg-Moore Monoids and Backtracking Monad Transformers , 2016, MSFP.

[6]  Chung-Kil Hur,et al.  A Provably Correct Sampler for Probabilistic Programs , 2015, FSTTCS.

[7]  Mauro Javier Jaskelioff.  Lifting of operations in modular monadic semantics , 2009 .

[8]  Ohad Kammar,et al.  Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints , 2016, 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[9]  David Tolpin,et al.  Probabilistic Programming in Anglican , 2015, ECML/PKDD.

[10]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[11]  J. H. Geuvers,et al.  Iteration and primitive recursion in categorical terms , 2007 .

[12]  G. Kelly.  A unified treatment of transfinite constructions for free algebras, free monoids, colimits, associated sheaves, and so on , 1980, Bulletin of the Australian Mathematical Society.

[13]  Jacques Carette,et al.  Probabilistic Inference by Program Transformation in Hakaru (System Description) , 2016, FLOPS.

[14]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[15]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[16]  Pat Hanrahan,et al.  Controlling procedural modeling programs with stochastically-ordered sequential Monte Carlo , 2015, ACM Trans. Graph..

[17]  Marcelo Fiore,et al.  List Objects with Algebraic Structure , 2017, FSCD.

[18]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[19]  Frank D. Wood,et al.  A New Approach to Probabilistic Programming Inference , 2014, AISTATS.

[20]  M. Erwig,et al.  Probabilistic Functional Programming in Haskell , 2005 .

[21]  Graham Hutton,et al.  A tutorial on the universality and expressiveness of fold , 1999, Journal of Functional Programming.

[22]  David Barber,et al.  Bayesian reasoning and machine learning , 2012 .

[23]  Thomas A. Henzinger,et al.  Probabilistic programming , 2014, FOSE.

[24]  Dustin Tran,et al.  Deep Probabilistic Programming , 2017, ICLR.

[25]  David Van Horn,et al.  Running Probabilistic Programs Backwards , 2015, ESOP.

[26]  Avi Pfeffer,et al.  Practical Probabilistic Programming , 2016, ILP.

[27]  Sam Staton,et al.  Commutative Semantics for Probabilistic Programming , 2017, ESOP.

[28]  Joseph T. Chang,et al.  Conditioning as disintegration , 1997 .

[29]  Nevin L. Zhang,et al.  A simple approach to Bayesian network computations , 1994 .

[30]  Thomas Ehrhard,et al.  Measurable Cones and Stable, Measurable Functions , 2018, POPL 2018.

[31]  Norman Ramsey,et al.  Exact Bayesian inference by symbolic disintegration , 2017, POPL.

[32]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[33]  Ohad Kammar,et al.  Denotational validation of higher-order Bayesian inference , 2017, Proc. ACM Program. Lang..

[34]  Eugenio Moggi,et al.  Notions of Computation and Monads , 1991, Inf. Comput..

[35]  C. Jones,et al.  A probabilistic powerdomain of evaluations , 1989, [1989] Proceedings. Fourth Annual Symposium on Logic in Computer Science.

[36]  Avi Pfeffer,et al.  IBAL: A Probabilistic Rational Programming Language , 2001, IJCAI.

[37]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Chung-chieh Shan,et al.  Embedded Probabilistic Programming , 2009, DSL.

[39]  Vikash K. Mansinghka,et al.  A stochastic programming perspective on nonparametric Bayes , 2008 .

[40]  Dan Roth,et al.  On the Hardness of Approximate Reasoning , 1993, IJCAI.

[41]  A. Kock.  Strong functors and monoidal monads , 1972 .

[42]  Daniel M. Roy,et al.  Noncomputable Conditional Distributions , 2011, 2011 IEEE 26th Annual Symposium on Logic in Computer Science.

[43]  Ugo Dal Lago,et al.  A lambda-calculus foundation for universal probabilistic programming , 2015, ICFP.

[44]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[45]  Zoubin Ghahramani,et al.  Turing: A Language for Flexible Probabilistic Inference , 2018 .

[46]  Ohad Kammar,et al.  Functional programming for modular Bayesian inference , 2018, Proc. ACM Program. Lang..

[47]  Andrew D. Gordon,et al.  Measure Transformer Semantics for Bayesian Machine Learning , 2011, ESOP.

[48]  Eric Moulines,et al.  Comparison of resampling schemes for particle filtering , 2005, ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005..

[49]  Sean Gerrish,et al.  Black Box Variational Inference , 2013, AISTATS.

[50]  Chung-Kil Hur,et al.  R2: An Efficient MCMC Sampler for Probabilistic Programs , 2014, AAAI.

[51]  A. Church.  The calculi of lambda-conversion , 1941 .

[52]  Christopher Strachey,et al.  Toward a mathematical semantics for computer languages , 1971 .

[53]  Paul Hudak,et al.  Modular Denotational Semantics for Compiler Construction , 1996, ESOP.

[54]  A. Kock.  COMMUTATIVE MONADS AS A THEORY OF DISTRIBUTIONS , 2011, 1108.5952.

[55]  David Huard,et al.  PyMC: Bayesian Stochastic Modelling in Python. , 2010, Journal of statistical software.

[56]  Liang Wang.  Owl: A General-Purpose Numerical Library in OCaml , 2017, ArXiv.

[57]  Matthias Felleisen,et al.  On the Expressive Power of Programming Languages , 1990, European Symposium on Programming.

[58]  Lawrence M. Murray.  Bayesian State-Space Modelling on High-Performance Hardware Using LibBi , 2013, 1306.3277.

[59]  Walter R. Gilks,et al.  A Language and Program for Complex Bayesian Modelling , 1994 .

[60]  Chung-chieh Shan,et al.  Composing Inference Algorithms as Program Transformations , 2016, UAI.

[61]  Radford M. Neal.  Probabilistic Inference Using Markov Chain Monte Carlo Methods , 2011 .

[62]  Manuel M. T. Chakravarty,et al.  Accelerating Haskell array codes with multicore GPUs , 2011, DAMP '11.

[63]  Judea Pearl,et al.  Reverend Bayes on Inference Engines: A Distributed Hierarchical Approach , 1982, AAAI.

[64]  A. Pettitt,et al.  Introduction to MCMC , 2012 .

[65]  Ohad Kammar,et al.  A convenient category for higher-order probability theory , 2017, 2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[66]  Bart Jacobs,et al.  From probability monads to commutative effectuses , 2018, J. Log. Algebraic Methods Program..

[67]  Howard Raiffa,et al.  Applied Statistical Decision Theory. , 1961 .

[68]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[69]  S. Duane,et al.  Hybrid Monte Carlo , 1987 .

[70]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[71]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[72]  Yura N. Perov,et al.  Venture: a higher-order probabilistic programming platform with programmable inference , 2014, ArXiv.

[73]  Christopher Strachey,et al.  Fundamental Concepts in Programming Languages , 2000, High. Order Symb. Comput..

[74]  P. J. Landin,et al.  The next 700 programming languages , 1966, CACM.

[75]  Frank D. Wood,et al.  Particle Gibbs with Ancestor Sampling for Probabilistic Programs , 2015, AISTATS.

[76]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[77]  Philip J. Scott,et al.  A categorical model for the geometry of interaction , 2006, Theor. Comput. Sci..

[78]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[79]  Dexter Kozen,et al.  Semantics of probabilistic programs , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[80]  Andrew Gelman,et al.  Handbook of Markov Chain Monte Carlo , 2011 .

[81]  Noah D. Goodman,et al.  Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation , 2011, AISTATS.

[82]  W. Gilks,et al.  Following a moving target—Monte Carlo inference for dynamic Bayesian models , 2001 .

[83]  R. Aumann.  Borel structures for function spaces , 1961 .

[84]  F. Marmolejo,et al.  MONADS AS EXTENSION SYSTEMS |NO ITERATION IS NECESSARY , 2010 .