Relational Marginal Problems: Theory and Estimation

In the propositional setting, the marginal problem is to find a (maximum-entropy) distribution that has some given marginals. We study this problem in a relational setting and make the following contributions. First, we compare two different notions of relational marginals. Second, we show a duality between the resulting relational marginal problems and the maximum likelihood estimation of the parameters of relational models, which generalizes a well-known duality from the propositional setting. Third, by exploiting the relational marginal formulation, we present a statistically sound method to learn the parameters of relational models that will be applied in settings where the number of constants differs between the training and test data. Furthermore, based on a relational generalization of marginal polytopes, we characterize cases where the standard estimators based on feature's number of true groundings needs to be adjusted and we quantitatively characterize the consequences of these adjustments. Fourth, we prove bounds on expected errors of the estimated parameters, which allows us to lower-bound, among other things, the effective sample size of relational training data.

[1]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[2]  Pranab Kumar Sen,et al.  On the Properties of U-Statistics When the Observations are not Independent , 1963 .

[3]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[4]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[5]  Joseph Y. Halpern,et al.  From Statistics to Beliefs , 1992, AAAI.

[6]  W. Hoeffding.  A Class of Statistics with Asymptotically Normal Distribution , 1948 .

[7]  Oliver Schulte,et al.  Challenge Paper: Marginal Probabilities for Instances and Classes (Poster Presentation SRL Workshop) , 2012 .

[8]  Jennifer Neville,et al.  Relational Learning with One Network: An Asymptotic Analysis , 2011, AISTATS.

[9]  A. Rinaldo,et al.  CONSISTENCY UNDER SAMPLING OF EXPONENTIAL RANDOM GRAPH MODELS. , 2011, Annals of statistics.

[10]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[11]  Guy Van den Broeck,et al.  Lifted generative learning of Markov logic networks , 2016, Machine Learning.

[12]  P. Diaconis,et al.  Estimating and understanding exponential random graph models , 2011, 1102.2650.

[13]  Mohit Singh,et al.  Entropy, optimization and counting , 2013, STOC.

[14]  D. Poole,et al.  Aggregation and Population Growth : The Relational Logistic Regression and Markov Logic Cases , 2012 .

[15]  Steven Schockaert,et al.  Induction of Interpretable Possibilistic Logic Theories from Relational Data , 2017, IJCAI.

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[18]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[19]  Michael Beetz,et al.  Extending Markov Logic to Model Probability Distributions in Relational Domains , 2007, KI.

[20]  Yuke Zhu,et al.  Modelling relational statistics with Bayes Nets , 2013, Machine Learning.