We propose to distinguish between deterministic conditioning, that is, conditioning on a sample from the joint data distribution, and stochastic conditioning, that is, conditioning on the distribution of the observable variable. Mostly, probabilistic programs follow the Bayesian approach by choosing a prior distribution of parameters and conditioning on observations. In a basic setting, individual observations are In a basic setting, individual observations are samples from the joint data distribution. However, observations may also be independent samples from marginal data distributions of each observable variable, summary statistics, or even data distributions themselves . These cases naturally appear in real life scenarios: samples from marginal distributions arise when different observations are collected by different parties, summary statistics (mean, variance, and quantiles) are often used to represent data collected over a large population, and data distributions may represent uncertainty during inference about future states of the world, that is, in planning. Probabilistic programming languages and frameworks which support conditioning on samples from the joint data distribution are not directly capable of expressing such models. We define the notion of stochastic conditioning and describe extensions of known general inference algorithms to probabilistic programs with stochastic conditioning. In case studies we provide probabilistic programs for several problems of statistical inference which are impossible or difficult to approach otherwise, perform inference on the programs, and analyse the results.
[1]
John K Kruschke,et al.
Bayesian data analysis.
,
2010,
Wiley interdisciplinary reviews. Cognitive science.
[2]
Sean Gerrish,et al.
Black Box Variational Inference
,
2013,
AISTATS.
[3]
Donald B. Rubin,et al.
A Case Study of the Robustness of Bayesian Methods of Inference: Estimating the Total in a Finite Population Using Transformations to Normality
,
1983
.
[4]
Mihalis Yannakakis,et al.
Shortest Paths Without a Map
,
1989,
Theor. Comput. Sci..
[5]
Dustin Tran,et al.
Automatic Differentiation Variational Inference
,
2016,
J. Mach. Learn. Res..
[6]
Chong Wang,et al.
Stochastic variational inference
,
2012,
J. Mach. Learn. Res..
[7]
Hongseok Yang,et al.
On Nesting Monte Carlo Estimators
,
2017,
ICML.
[8]
Armando Solar-Lezama,et al.
The Random Conditional Distribution for Higher-Order Probabilistic Inference
,
2019,
ArXiv.
[9]
David Tolpin,et al.
Black-Box Policy Search with Probabilistic Programs
,
2015,
AISTATS.
[10]
Noah D. Goodman,et al.
Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation
,
2011,
AISTATS.
[11]
Tianqi Chen,et al.
A Complete Recipe for Stochastic Gradient MCMC
,
2015,
NIPS.
[12]
Tom Rainforth,et al.
Nesting Probabilistic Programs
,
2018,
UAI.