BLOG: Probabilistic Models with Unknown Objects

This paper introduces and illustrates BLOG, a formal language for defining probability models over worlds with unknown objects and identity uncertainty. BLOG unifies and extends several existing approaches. Subject to certain acyclicity constraints, every BLOG model specifies a unique probability distribution over first-order model structures that can contain varying and unbounded numbers of objects. Furthermore, complete inference algorithms exist for a large fragment of the language. We also introduce a probabilistic form of Skolemization for handling evidence.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  H B NEWCOMBE,et al.  Automatic linkage of vital records. , 1959, Science.

[3]  H. Gaifman.  Concerning measures in first order calculi , 1964 .

[4]  Robert W. Sittler,et al.  An Optimal Data Association Problem in Surveillance Theory , 1964, IEEE Transactions on Military Electronics.

[5]  J. Heijenoort.  From Frege to Gödel: A Source Book in Mathematical Logic, 1879-1931 , 1967 .

[6]  C. Nash-Williams,et al.  Infinite graphs—A survey , 1967 .

[7]  Ivan P. Fellegi,et al.  A Theory for Record Linkage , 1969 .

[8]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[9]  Herbert B. Enderton,et al.  A mathematical introduction to logic , 1972 .

[10]  Lalit R. Bahl,et al.  Decoding for channels with insertions, deletions, and substitutions with applications to speech recognition , 1975, IEEE Trans. Inf. Theory.

[11]  D. Reid.  An algorithm for tracking multiple targets , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[12]  P. Billingsley,et al.  Probability and Measure , 1980 .

[13]  Rudolf Mathon,et al.  A Note on the Graph Isomorphism counting Problem , 1979, Inf. Process. Lett..

[14]  Raymond Reiter,et al.  Equality and Domain Closure in First-Order Databases , 1980, JACM.

[15]  Lalit R. Bahl,et al.  A Maximum Likelihood Approach to Continuous Speech Recognition , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  T. Ferguson.  BAYESIAN DENSITY ESTIMATION BY MIXTURES OF NORMAL DISTRIBUTIONS , 1983 .

[17]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  T. Speed,et al.  Recursive causal models , 1984, Journal of the Australian Mathematical Society. Series A. Pure Mathematics and Statistics.

[19]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[20]  Ross D. Shachter.  Evaluating Influence Diagrams , 1986, Oper. Res..

[21]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[22]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[23]  Hans-Otto Georgii,et al.  Gibbs Measures and Phase Transitions , 1988 .

[24]  Stephen Muggleton,et al.  Machine Invention of First Order Predicates by Inverting Resolution , 1988, ML.

[25]  Ross D. Shachter,et al.  Simulation Approaches to General Probabilistic Inference on Belief Networks , 2013, UAI.

[26]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[27]  Joseph Y. Halpern.  An Analysis of First-Order Logics of Probability , 1989, IJCAI.

[28]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[29]  Kuo-Chu Chang,et al.  Weighing and Integrating Evidence for Stochastic Simulation in Bayesian Networks , 2013, UAI.

[30]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[31]  Ross D. Shachter,et al.  Symbolic Probabilistic Inference in Belief Networks , 1990, AAAI.

[32]  R. Durrett.  Probability: Theory and Examples , 1993 .

[33]  Steffen L. Lauritzen,et al.  Independence properties of directed markov fields , 1990, Networks.

[34]  V. S. Subrahmanian,et al.  Probabilistic Logic Programming , 1992, Inf. Comput..

[35]  J. Q. Smith,et al.  1. Bayesian Statistics 4 , 1993 .

[36]  Robert P. Goldman,et al.  A Bayesian Model of Plan Recognition , 1993, Artif. Intell..

[37]  David Poole,et al.  Probabilistic Horn Abduction and Bayesian Networks , 1993, Artif. Intell..

[38]  Walter R. Gilks,et al.  A Language and Program for Complex Bayesian Modelling , 1994 .

[39]  Nevin L. Zhang,et al.  A simple approach to Bayesian network computations , 1994 .

[40]  Shalom Lappin,et al.  An Algorithm for Pronominal Anaphora Resolution , 1994, CL.

[41]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[42]  Walter R. Gilks,et al.  Introduction to general state-space Markov chain theory , 1995 .

[43]  William A. Gale,et al.  Good-Turing Frequency Estimation Without Tears , 1995, J. Quant. Linguistics.

[44]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[45]  David Heckerman,et al.  Knowledge Representation and Inference in Similarity Networks and Bayesian Multinets , 1996, Artif. Intell..

[46]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[47]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[48]  G. Casella,et al.  Rao-Blackwellisation of sampling schemes , 1996 .

[49]  K. Ciesielski.  Set Theory for the Working Mathematician , 1997 .

[50]  David Poole,et al.  The Independent Choice Logic for Modelling Multiple Agents Under Uncertainty , 1997, Artif. Intell..

[51]  Manfred Jaeger,et al.  Relational Bayesian Networks , 1997, UAI.

[52]  P. Green,et al.  Corrigendum: On Bayesian analysis of mixtures with an unknown number of components , 1997 .

[53]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[54]  C. Lee Giles,et al.  CiteSeer: an automatic citation indexing system , 1998, DL '98.

[55]  Avi Pfeffer,et al.  Probabilistic Frame-Based Systems , 1998, AAAI/IAAI.

[56]  Manfred Jaeger,et al.  Reasoning About Infinite Random Structures with Relational Bayesian Networks , 1998, KR.

[57]  C. Lee Giles,et al.  Autonomous citation matching , 1999, AGENTS '99.

[58]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[59]  C. Lee Giles,et al.  Digital Libraries and Autonomous Citation Indexing , 1999, Computer.

[60]  Rina Dechter,et al.  Bucket Elimination: A Unifying Framework for Reasoning , 1999, Artif. Intell..

[61]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[62]  Thomas Lukasiewicz,et al.  Probalilistic Logic Programming under Maximum Entropy , 1999, ESCQARU.

[63]  Daphne Koller,et al.  Probabilistic reasoning for complex systems , 1999 .

[64]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[65]  Luc De Raedt,et al.  Bayesian Logic Programs , 2001, ILP Work-in-progress reports.

[66]  Andrew McCallum,et al.  Efficient clustering of high-dimensional data sets with application to reference matching , 2000, KDD '00.

[67]  Avi Pfeffer,et al.  Semantics and Inference for Recursive Probability Models , 2000, AAAI/IAAI.

[68]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[69]  Ben Taskar,et al.  Learning Probabilistic Models of Relational Structure , 2001, ICML.

[70]  Luc De Raedt,et al.  Adaptive Bayesian Logic Programs , 2001, ILP.

[71]  Avi Pfeffer,et al.  IBAL: A Probabilistic Rational Programming Language , 2001, IJCAI.

[72]  Stuart J. Russell,et al.  Approximate inference for first-order probabilistic languages , 2001, IJCAI.

[73]  Hwee Tou Ng,et al.  A Machine Learning Approach to Coreference Resolution of Noun Phrases , 2001, CL.

[74]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[75]  Yoshitaka Kameya,et al.  Parameter Learning of Logic Programs for Symbolic-Statistical Modeling , 2001, J. Artif. Intell. Res..

[76]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[77]  Stuart J. Russell,et al.  Identity Uncertainty and Citation Matching , 2002, NIPS.

[78]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[79]  Zhuowen Tu,et al.  Image Segmentation by Data-Driven Markov Chain Monte Carlo , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[80]  S. T. Buckland,et al.  Estimating Animal Abundance , 2002 .

[81]  Ben Taskar,et al.  Learning Probabilistic Models of Link Structure , 2003, J. Mach. Learn. Res..

[82]  Andrew McCallum,et al.  Toward Conditional Models of Identity Uncertainty with Application to Proper Noun Coreference , 2003, IIWeb.

[83]  Aymeric Puech.  A Comparison of Stochastic Logic Programs and Bayesian Logic Programs , 2003 .

[84]  David Poole,et al.  First-order probabilistic inference , 2003, IJCAI.

[85]  Michael I. Jordan,et al.  A generalized mean field algorithm for variational inference in exponential families , 2002, UAI.

[86]  David M. Pennock,et al.  Statistical relational learning for document mining , 2003, Third IEEE International Conference on Data Mining.

[87]  Nevin Lianwen Zhang,et al.  Exploiting Contextual Independence In Probabilistic Inference , 2011, J. Artif. Intell. Res..

[88]  Stuart J. Russell,et al.  BLOG: Relational Modeling with Unknown Objects , 2004 .

[89]  Dan Roth,et al.  Robust Reading: Identification and Tracing of Ambiguous Names , 2004, NAACL.

[90]  David Heckerman,et al.  Probabilistic Models for Relational Data , 2004 .

[91]  Manfred Jaeger,et al.  Complex Probabilistic Modeling with Recursive Relational Bayesian Networks , 2001, Annals of Mathematics and Artificial Intelligence.

[92]  Andrew McCallum,et al.  An Integrated, Conditional Model of Information Extraction and Coreference with Appli , 2004, UAI.

[93]  Maurice Bruynooghe,et al.  Logic programs with annotated disjunctions , 2004, NMR.

[94]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[95]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[96]  Radford M. Neal,et al.  A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model , 2004 .

[97]  Songhwai Oh,et al.  Markov chain Monte Carlo data association for general multiple-target tracking problems , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[98]  E. Mjolsness.  Labeled graph notations for graphical models , 2004 .

[99]  Andrew McCallum,et al.  Conditional Models of Identity Uncertainty with Application to Noun Coreference , 2004, NIPS.

[100]  Paulo Cesar G. da Costa,et al.  Of Starships and Klingons: Bayesian Logic for the 23rd Century , 2005, UAI.

[101]  Pedro M. Domingos,et al.  Object Identification with Attribute-Mediated Dependences , 2005, PKDD.

[102]  Henry A. Kautz,et al.  Performing Bayesian Inference by Weighted Model Counting , 2005, AAAI.

[103]  Stuart J. Russell,et al.  Approximate Inference for Infinite Contingent Bayesian Networks , 2005, AISTATS.

[104]  Andrew McCallum,et al.  Joint deduplication of multiple record types in relational data , 2005, CIKM '05.

[105]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[106]  Nir Friedman,et al.  Learning Hidden Variable Networks: The Information Bottleneck Approach , 2005, J. Mach. Learn. Res..

[107]  Dan Roth,et al.  Lifted First-Order Probabilistic Inference , 2005, IJCAI.

[108]  Nando de Freitas,et al.  Nonparametric Bayesian Logic , 2005, UAI.

[109]  Stuart J. Russell,et al.  General-Purpose MCMC Inference over Relational Structures , 2006, UAI.

[110]  Pedro M. Domingos,et al.  Entity Resolution with Markov Logic , 2006, Sixth International Conference on Data Mining (ICDM'06).

[111]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[112]  Manfred Jaeger,et al.  Compiling relational Bayesian networks for exact inference , 2006, Int. J. Approx. Reason..

[113]  Eric Mjolsness,et al.  Stochastic Process Semantics for Dynamical Grammar Syntax: An Overview , 2005, AI&M.

[114]  Kathryn B. Laskey.  MEBN: A Logic for Open-World Probabilistic Reasoning , 2006 .

[115]  Ben Taskar,et al.  Markov Logic: A Unifying Framework for Statistical Relational Learning , 2007 .

[116]  Ben Taskar,et al.  BLOG: Probabilistic Models with Unknown Objects , 2007 .