General-Purpose MCMC Inference over Relational Structures

Tasks such as record linkage and multi-target tracking, which involve reconstructing the set of objects that underlie some observed data, are particularly challenging for probabilistic inference. Recent work has achieved efficient and accurate inference on such problems using Markov chain Monte Carlo (MCMC) techniques with customized proposal distributions. Currently, implementing such a system requires coding MCMC state representations and acceptance probability calculations that are specific to a particular application. An alternative approach, which we pursue in this paper, is to use a general-purpose probabilistic modeling language (such as BLOG) and a generic Metropolis-Hastings MCMC algorithm that supports user-supplied proposal distributions. Our algorithm gains flexibility by using MCMC states that are only partial descriptions of possible worlds; we provide conditions under which MCMC over partial worlds yields correct answers to queries. We also show how to use a context-specific Bayes net to identify the factors in the acceptance probability that need to be computed for a given proposed move. Experimental results on a citation matching task show that our general-purpose MCMC engine compares favorably with an application-specific system.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  Ivan P. Fellegi,et al.  A Theory for Record Linkage , 1969 .

[3]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[4]  Rudolf Mathon,et al.  A Note on the Graph Isomorphism counting Problem , 1979, Inf. Process. Lett..

[5]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[6]  Walter R. Gilks,et al.  A Language and Program for Complex Bayesian Modelling , 1994 .

[7]  C. Lee Giles,et al.  Autonomous citation matching , 1999, AGENTS '99.

[8]  Andrew McCallum,et al.  Efficient clustering of high-dimensional data sets with application to reference matching , 2000, KDD '00.

[9]  James Cussens,et al.  Markov Chain Monte Carlo using Tree-Based Priors on Model Structure , 2001, UAI.

[10]  Avi Pfeffer,et al.  IBAL: A Probabilistic Rational Programming Language , 2001, IJCAI.

[11]  Stuart J. Russell,et al.  Identity Uncertainty and Citation Matching , 2002, NIPS.

[12]  Manfred Jaeger,et al.  Complex Probabilistic Modeling with Recursive Relational Bayesian Networks , 2001, Annals of Mathematics and Artificial Intelligence.

[13]  Andrew McCallum,et al.  An Integrated, Conditional Model of Information Extraction and Coreference with Appli , 2004, UAI.

[14]  Radford M. Neal,et al.  A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model , 2004 .

[15]  Songhwai Oh,et al.  Markov chain Monte Carlo data association for general multiple-target tracking problems , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[16]  E. Mjolsness,et al.  Variable-Structure Systems from Graphs and Grammars , 2005 .

[17]  Paulo Cesar G. da Costa,et al.  Of Starships and Klingons: Bayesian Logic for the 23rd Century , 2005, UAI.

[18]  Manfred Jaeger.  Importance Sampling on Relational Bayesian Networks , 2005, Probabilistic, Logical and Relational Learning.

[19]  Stuart J. Russell,et al.  Approximate Inference for Infinite Contingent Bayesian Networks , 2005, AISTATS.

[20]  James Cussens,et al.  Exploiting Informative Priors for Bayesian Classification and Regression Trees , 2005, IJCAI.

[21]  Stuart J. Russell,et al.  BLOG: Probabilistic Models with Unknown Objects , 2005, IJCAI.

[22]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.