TensorFlow Distributions

The TensorFlow Distributions library implements a vision of probability theory adapted to the modern deep-learning paradigm of end-to-end differentiable computation. Building on two basic abstractions, it offers flexible building blocks for probabilistic computation. Distributions provide fast, numerically stable methods for generating samples and computing statistics, e.g., log density. Bijectors provide composable volume-tracking transformations with automatic caching. Together these enable modular construction of high dimensional distributions and transformations not possible with previous libraries (e.g., pixelCNNs, autoregressive flows, and reversible residual networks). They are the workhorse behind deep probabilistic programming systems like Edward and empower fast black-box inference in probabilistic models built on deep-network components. TensorFlow Distributions has proven an important part of the TensorFlow toolkit within Google and in the broader deep learning community.

[1]  Nick Golding,et al.  greta: simple and scalable statistical modelling in R , 2019, J. Open Source Softw..

[2]  D. Blei,et al.  Implicit Causal Models for Genome-wide Association Studies , 2017, ICLR.

[3]  Jun Zhu,et al.  ZhuSuan: A Library for Bayesian Deep Learning , 2017, arXiv.org.

[4]  Iain Murray,et al.  Masked Autoregressive Flow for Density Estimation , 2017, NIPS.

[5]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[6]  Diederik P. Kingma,et al.  PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications , 2017, ICLR.

[7]  Dustin Tran,et al.  Deep Probabilistic Programming , 2017, ICLR.

[8]  Marcus A. Brubaker,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[9]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[10]  Yee Whye Teh,et al.  The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables , 2016, ICLR.

[11]  Dustin Tran,et al.  Edward: A library for probabilistic modeling, inference, and criticism , 2016, ArXiv.

[12]  Scott W. Linderman,et al.  Reparameterization Gradients through Acceptance-Rejection Sampling Algorithms , 2016, AISTATS.

[13]  Alán Aspuru-Guzik,et al.  Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules , 2016, ACS central science.

[14]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[15]  Max Welling,et al.  Improved Variational Inference with Inverse Autoregressive Flow , 2016, NIPS 2016.

[16]  Ryan P. Adams,et al.  Composing graphical models with neural networks for structured representations and fast inference , 2016, NIPS.

[17]  Dustin Tran,et al.  Automatic Differentiation Variational Inference , 2016, J. Mach. Learn. Res..

[18]  Koray Kavukcuoglu,et al.  Pixel Recurrent Neural Networks , 2016, ICML.

[19]  Bob Carpenter,et al.  The Stan Math Library: Reverse-Mode Automatic Differentiation in C++ , 2015, ArXiv.

[20]  Pieter Abbeel,et al.  Gradient Estimation Using Stochastic Computation Graphs , 2015, NIPS.

[21]  Christian Osendorfer,et al.  Learning Stochastic Recurrent Networks , 2014, NIPS 2014.

[22]  Sean Gerrish,et al.  Black Box Variational Inference , 2013, AISTATS.

[23]  Diederik P. Kingma,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[24]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[25]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[26]  Edward McCrorie,et al.  Black , 2011 .

[27]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[28]  이성수,et al.  Simulation , 2006, Healthcare Simulation at a Glance.

[29]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[30]  Larry Wasserman,et al.  All of Statistics: A Concise Course in Statistical Inference , 2004 .

[31]  A. Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[32]  George Marsaglia,et al.  A simple method for generating gamma variables , 2000, TOMS.

[33]  A. Müller.  Integral Probability Metrics and Their Generating Classes of Functions , 1997, Advances in Applied Probability.

[34]  Ross Ihaka,et al.  Gentleman R: R: A language for data analysis and graphics , 1996 .

[35]  Peter J. Bickel,et al.  S: An Interactive Environment for Data Analysis and Graphics , 1984 .

[36]  M. Spivak.  A comprehensive introduction to differential geometry , 1979 .

[37]  H. Niederreiter.  Quasi-Monte Carlo methods and pseudo-random numbers , 1978 .

[38]  T. Morimoto.  Markov Processes and the H -Theorem , 1963 .

[39]  M. E. Muller,et al.  A Note on the Generation of Random Normal Deviates , 1958 .

[40]  Jinsung Yoon,et al.  GENERATIVE ADVERSARIAL NETS , 2018 .

[41]  Daniel M. Roy,et al.  EXCHANGEABLE RANDOM PRIMITIVES , 2015 .

[42]  Yezhou Huang.  Correlated Topic Models , 2014 .

[43]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[44]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[45]  D. Aldous.  Exchangeability and related topics , 1985 .

[46]  Robert C. Williamson,et al.  25th Annual Conference on Learning Theory Divergences and Risks for Multiclass Experiments , 2022 .