A programming language for probabilistic computation

As probabilistic computations play an increasing role in solving various problems, researchers have designed probabilistic languages to facilitate their modeling. Most of the existing probabilistic languages, however, focus only on discrete distributions, and there has been little effort to develop probabilistic languages whose expressive power is beyond discrete distributions. This dissertation presents a probabilistic language, called PTP (ProbabilisTic Programming), which supports all kinds of probability distributions. The key idea behind PTP is to use sampling functions, i.e., mappings from the unit interval (0.0,1.0] to probability domains, to specify probability distributions. By using sampling functions as its mathematical basis, PTP provides a unified representation scheme for probability distributions, without drawing a syntactic or semantic distinction between different kinds of probability distributions. Independently of PTP, we develop a linguistic framework, called λ c, to account for computational effects in general. λ c extends a monadic language by applying the possible world interpretation of modal logic. A characteristic feature of λc is the distinction between stateful computational effects, called world effects , and contextual computational effects, called control effects . PTP arises as an instance of λc with a language construct for probabilistic choices. We use a sound and complete translator of PTP to embed it in Objective CAML. The use of PTP is demonstrated with three applications in robotics: robot localization, people tracking, and robotic mapping. Thus PTP serves as another example of high-level language applied to a problem domain where imperative languages have been traditionally dominant.

[1]  Guy E. Blelloch,et al.  Selective memoization , 2003, POPL '03.

[2]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[3]  Philip Wadler,et al.  Combining Monads , 1992, Functional Programming.

[4]  Simon L. Peyton Jones,et al.  Lazy functional state threads , 1994, PLDI '94.

[5]  Christoph Lüth,et al.  Composing monads using coproducts , 2002, ICFP '02.

[6]  Philip Wadler,et al.  The marriage of effects and monads , 1998, ICFP '98.

[7]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[8]  Evangelos E. Milios,et al.  Globally Consistent Range Scan Alignment for Environment Mapping , 1997, Auton. Robots.

[9]  Aleksandar Nanevski,et al.  From dynamic binding to state via modal possibility , 2003, PPDP '03.

[10]  Amr Sabry,et al.  Correctness of monadic state: an imperative call-by-need calculus , 1998, POPL '98.

[11]  Peyton Jones,et al.  Haskell 98 language and libraries : the revised report , 2003 .

[12]  Michael Mendler,et al.  Propositional Lax Logic , 1997, Inf. Comput..

[13]  John Launchbury,et al.  Recursive monadic bindings , 2000, ICFP '00.

[14]  Guy E. Blelloch,et al.  Adaptive functional programming , 2002, POPL '02.

[15]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[16]  Frank Pfenning,et al.  A type theory for memory allocation and data layout , 2003, POPL '03.

[17]  R. Kent Dybvig,et al.  Revised5 Report on the Algorithmic Language Scheme , 1986, SIGP.

[18]  Norman Ramsey,et al.  Stochastic lambda calculus and monads of probability distributions , 2002, POPL '02.

[19]  Günter Asser,et al.  Zeitschrift für mathematische Logik und Grundlagen der Mathematik , 1955 .

[20]  Sungwoo Park,et al.  A calculus for probabilistic languages , 2003, TLDI '03.

[21]  Dexter Kozen,et al.  Semantics of probabilistic programs , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[22]  Linus Schrage,et al.  A guide to simulation , 1983 .

[23]  Aleksandar Nanevski A modal calculus for effect handling , 2003 .

[24]  Lawrence R. Rabiner,et al.  A tutorial on Hidden Markov Models , 1986 .

[25]  Simon L. Peyton Jones,et al.  A semantics for imprecise exceptions , 1999, PLDI '99.

[26]  Sebastian Thrun,et al.  Towards programming tools for robots that integrate probabilistic computation and learning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[27]  W. Burgard,et al.  Markov Localization for Mobile Robots in Dynamic Environments , 1999, J. Artif. Intell. Res..

[28]  Derek Dreyer,et al.  A type system for well-founded recursion , 2004, POPL.

[29]  Sebastian Thrun,et al.  Probabilistic Algorithms in Robotics , 2000, AI Mag..

[30]  Timothy G. Griffin,et al.  A formulae-as-type notion of control , 1989, POPL '90.

[31]  Nick Benton,et al.  Computational types from a logical perspective , 1998, Journal of Functional Programming.

[32]  Simon L. Peyton Jones,et al.  State in Haskell , 1995, LISP Symb. Comput..

[33]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[34]  Gérard Boudol The Recursive Record Semantics of Objects Revisited , 2001, ESOP.

[35]  Robert Harper,et al.  Typing first-class continuations in ML , 1991, POPL '91.

[36]  Sebastian Thrun,et al.  FastSLAM: a factored solution to the simultaneous localization and mapping problem , 2002, AAAI/IAAI.

[37]  William Whittaker,et al.  Conditional particle filters for simultaneous mobile robot localization and people-tracking , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[38]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[39]  N. Saheb-Djahromi,et al.  Probabilistic LCF , 1978, International Symposium on Mathematical Foundations of Computer Science.

[40]  Sebastian Thrun,et al.  Robotic mapping: a survey , 2003 .

[41]  Amr Sabry,et al.  An Abstract Monadic Semantics for Value Recursion , 2004, FICS.

[42]  Frank Pfenning,et al.  A judgmental reconstruction of modal logic , 2001, Mathematical Structures in Computer Science.

[43]  Alex K. Simpson,et al.  The proof theory and semantics of intuitionistic modal logic , 1994 .

[44]  Philip Wadler,et al.  Comprehending monads , 1990, Mathematical Structures in Computer Science.

[45]  Eugenio Moggi,et al.  Notions of Computation and Monads , 1991, Inf. Comput..

[46]  Avi Pfeffer,et al.  IBAL: A Probabilistic Rational Programming Language , 2001, IJCAI.

[47]  David Dillon,et al.  Language learning , 2006, Language Teaching.

[48]  Amr Sabry,et al.  Monadic encapsulation in ML , 1999, ICFP '99.

[49]  Torben Æ. Mogensen Roll : A Language for Specifying Die-Rolls , 2003, PADL.

[50]  Radha Jagadeesan,et al.  Stochastic processes as concurrent constraint programs , 1999, POPL '99.

[51]  Amr Sabry,et al.  What is a purely functional language? , 1998, Journal of Functional Programming.

[52]  John T. Gill,et al.  Computational complexity of probabilistic Turing machines , 1974, STOC '74.

[53]  Amr Sabry,et al.  Monadic encapsulation of effects: a revised approach (extended version) , 2001, J. Funct. Program..

[54]  William A. Howard,et al.  The formulae-as-types notion of construction , 1969 .

[55]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[56]  Frank Pfenning,et al.  A monadic analysis of information flow security with mutable state , 2005, J. Funct. Program..

[57]  J. L. Roux An Introduction to the Kalman Filter , 2003 .

[58]  Claire Jones,et al.  Probabilistic non-determinism , 1990 .

[59]  David Walker,et al.  An effective theory of type refinements , 2003, ICFP '03.

[60]  Philip Wadler,et al.  A reflection on call-by-value , 1997, TOPL.

[61]  Eugenio Moggi,et al.  Computational lambda-calculus and monads , 1989, [1989] Proceedings. Fourth Annual Symposium on Logic in Computer Science.

[62]  Abbas Edalat,et al.  A new representation for exact real numbers , 1997, MFPS.

[63]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[64]  Simon L. Peyton Jones,et al.  Concurrent Haskell , 1996, POPL '96.

[65]  Philip Wadler,et al.  The essence of functional programming , 1992, POPL '92.

[66]  David A. McAllester,et al.  Effective Bayesian Inference for Stochastic Programs , 1997, AAAI/IAAI.

[67]  George F. Luger,et al.  Toward General Analysis of Recursive Probability Models , 2001, UAI.

[68]  Sebastian Thrun,et al.  A Programming Language Extension for Probabilistic Robot Programming , 2000 .

[69]  Simon Peyton Jones,et al.  Tackling the Awkward Squad: monadic input/output, concurrency, exceptions, and foreign-language calls in Haskell , 2005 .

[70]  Paul Bratley,et al.  A guide to simulation , 1983 .