A Domain-Theoretic Approach to Statistical Programming Languages

We give a domain-theoretic semantics to a statistical programming language, using the plain old category of dcpos, in contrast to some more sophisticated recent proposals. Remarkably, our monad of minimal valuations is commutative, which allows for program transformations that permute the order of independent random draws, as one would expect. A similar property is not known for Jones and Plotkin’ s monad of continuous valuations. Instead of working with true real numbers, we work with exact real arithmetic, providing a bridge towards possible implementations. (Implementations by themselves are not addressed here.) Rather remarkably, we show that restricting ourselves to minimal valuations does not restrict us much: all measures on the real line can be modeled by minimal valuations on the domain IR⊥ of exact real arithmetic. We give three operational semantics for our language, and we show that they are all adequate with respect to the denotational semantics. We also explore quite a few examples in order to demonstrate that our semantics computes exactly as one would expect, and in order to debunk the myth that a semantics based on continuous maps would not be expressive enough to encode measures with non-compact support using only measures with compact support, or to encode measures via non-continuous density functions, for instance. 1 ar X iv :2 10 6. 16 19 0v 2 [ cs .L O ] 1 3 Ju l 2 02 1

[1]  Samson Abramsky,et al.  Domain theory , 1995, LICS 1995.

[2]  Matthew de Brecht Quasi-Polish spaces , 2011, Ann. Pure Appl. Log..

[3]  T. A. Bray,et al.  A Convenient Method for Generating Normal Variables , 1964 .

[4]  Abbas Edalat,et al.  Integration in Real PCF , 2000, Inf. Comput..

[5]  Dana S. Scott,et al.  A Type-Theoretical Alternative to ISWIM, CUCH, OWHY , 1993, Theor. Comput. Sci..

[6]  A. Kock COMMUTATIVE MONADS AS A THEORY OF DISTRIBUTIONS , 2011, 1108.5952.

[7]  Andrew M. Pitts,et al.  Relational Properties of Domains , 1996, Inf. Comput..

[8]  Philipp Sünderhauf A Faithful Computational Model of the Real Numbers , 1995, Theor. Comput. Sci..

[9]  David R. Lester Vuillemin's Exact Real Arithmetic , 1991, Functional Programming.

[10]  Gordon D. Plotkin,et al.  An axiomatisation of computationally adequate domain theoretic models of FPC , 1994, Proceedings Ninth Annual IEEE Symposium on Logic in Computer Science.

[11]  Claire Jones,et al.  Probabilistic non-determinism , 1990 .

[12]  Jean Goubault-Larrecq,et al.  Domain-complete and LCS-complete Spaces , 2019, ISDT.

[13]  Thomas Ehrhard,et al.  Measurable cones and stable, measurable functions: a model for probabilistic higher-order programming , 2017, Proc. ACM Program. Lang..

[14]  Achim Jung,et al.  The troublesome probabilistic powerdomain , 1997, COMPROX.

[15]  Norman Ramsey,et al.  Exact Bayesian inference by symbolic disintegration , 2017, POPL.

[16]  Reinhold Heckmann,et al.  Spaces of Valuations , 1996 .

[17]  Holger Schulz,et al.  Type two theory of effectivity and Real PCF , 1998, Workshop on Domains.

[18]  J. Norris Appendix: probability and measure , 1997 .

[19]  τ-smooth BOREL Measures on Topological Spaces , 1977 .

[20]  Roman Fric,et al.  A Categorical Approach to Probability Theory , 2010, Stud Logica.

[21]  Jean Goubault-Larrecq A Probabilistic and Non-Deterministic Call-by-Push-Value Language , 2019, 2019 34th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[22]  J. Isbell,et al.  Product spaces in locales , 1981 .

[23]  Ohad Kammar,et al.  Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints , 2016, 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[24]  Martín Hötzel Escardó,et al.  PCF Extended with Real Numbers , 1996, Theor. Comput. Sci..

[25]  Mathieu Hoyrup,et al.  Effective symbolic dynamics, random points, statistical behavior, complexity and entropy , 2007, Inf. Comput..

[26]  Ugo Dal Lago,et al.  The Geometry of Bayesian Programming , 2019, 2019 34th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[27]  Matthias Schröder,et al.  Representing probability measures using probabilistic processes , 2006, J. Complex..

[28]  M. E. Muller,et al.  A Note on the Generation of Random Normal Deviates , 1958 .

[29]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[30]  Abbas Edalat,et al.  Semantics of exact real arithmetic , 1997, Proceedings of Twelfth Annual IEEE Symposium on Logic in Computer Science.

[31]  Klaus Weihrauch,et al.  Computability on the probability measures on the Borel sets of the unit interval , 1997, CCA.

[32]  Weng Kin Ho Exact real calculator for everyone , 2013 .

[33]  Baodian Wei,et al.  On Generating Exponentially Distributed Variates by Using Early Rejection , 2019, 2019 IEEE 5th International Conference on Computer and Communications (ICCC).

[34]  José Raymundo Marcial-Romero,et al.  Semantics of a sequential language for exact real-number computation , 2004, Proceedings of the 19th Annual IEEE Symposium on Logic in Computer Science, 2004..

[35]  Eugenio Moggi,et al.  Computational lambda-calculus and monads , 1989, [1989] Proceedings. Fourth Annual Symposium on Logic in Computer Science.

[36]  C. Jones,et al.  A probabilistic powerdomain of evaluations , 1989, [1989] Proceedings. Fourth Annual Symposium on Logic in Computer Science.

[37]  Ohad Kammar,et al.  A convenient category for higher-order probability theory , 2017, 2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[38]  Accademia nazionale dei Lincei. Classe di scienze fisiche,et al.  Atti della Accademia nazionale dei Lincei. Classe di scienze fisiche, matematiche e naturali. Rendiconti. Serie ottava , 1946 .

[39]  Michael Mislove,et al.  Completing Simple Valuations in K-categories , 2020, Topology and its Applications.

[40]  Jacques Carette,et al.  Probabilistic Inference by Program Transformation in Hakaru (System Description) , 2016, FLOPS.

[41]  Jean Goubault-Larrecq Full abstraction for non-deterministic and probabilistic extensions of PCF I: The angelic cases , 2015, J. Log. Algebraic Methods Program..

[42]  Hans-Juergen Boehm,et al.  Exact real arithmetic: a case study in higher order programming , 1986, LFP '86.

[43]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[44]  V. E. Cazanescu Algebraic theories , 2004 .

[45]  Frank D. Wood,et al.  A New Approach to Probabilistic Programming Inference , 2014, AISTATS.

[46]  Jimmie D. Lawson,et al.  Spaces of maximal points , 1997, Mathematical Structures in Computer Science.

[47]  G. Forsythe Von Neumann''s comparison method for random sampling from the normal and other distributions. , 1972 .

[48]  Sam Staton,et al.  Commutative Semantics for Probabilistic Programming , 2017, ESOP.

[49]  Martín Hötzel Escardó,et al.  PCF extended with real numbers : a domain-theoretic approach to higher-order exact real number computation , 1997 .

[50]  Daniel M. Roy Computability, inference and modeling in probabilistic programming , 2011 .

[51]  Vladimir Zamdzhiev,et al.  Commutative Monads for Probabilistic Programming Languages , 2021, 2021 36th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[52]  Ohad Kammar,et al.  A domain theory for statistical probabilistic programming , 2018, Proc. ACM Program. Lang..

[53]  Yura N. Perov,et al.  Venture: a higher-order probabilistic programming platform with programmable inference , 2014, ArXiv.

[54]  D. Plume A calculator for exact real number computation , 1998 .

[55]  Daniel M. Roy,et al.  Noncomputable Conditional Distributions , 2011, 2011 IEEE 26th Annual Symposium on Logic in Computer Science.

[56]  Klaus Keimel,et al.  D-completions and the d-topology , 2009, Ann. Pure Appl. Log..