Learning Rules from Incomplete Examples: A Pragmatic Approach

In this paper, we consider the problem of inductively learning rules from specific facts extracted from texts. This problem is challenging due to two reasons. First, natural texts areradically incompletesince there are always too many facts to mention. Second, natural texts aresystematically biasedtowards novelty and surprise, which presents an unrepresentative sample to the learner. Our solutions to these two problems are based on building a generative observation model of what is mentioned and what is extracted given what is true. We first present a Multiple-predicate Bootstrappingapproach that consists of iteratively learning if-then rules based on an implicit observation model and then imputing new facts implied by the learned rules. Second, we present an iterative ensemble colearning approach, where multiple decisiontrees are learned from bootstrap samples of the incomplete training data, and facts are imputed based on weighted majority.

[1]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[2]  D. Rubin,et al.  Multiple Imputation for Nonresponse in Surveys , 1989 .

[3]  D. Rubin.  Multiple imputation for nonresponse in surveys , 1989 .

[4]  Estevam R. Hruschka,et al.  Coupled semi-supervised learning for information extraction , 2010, WSDM '10.

[5]  J. Schafer,et al.  Missing data: our view of the state of the art. , 2002, Psychological methods.

[6]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[7]  Luc De Raedt,et al.  Multiple Predicate Learning in Two Inductive Logic Programming Settings , 1996, Log. J. IGPL.

[8]  Joost N. Kok,et al.  Efficient Frequent Query Discovery in FARMER , 2003, PKDD.

[9]  R. Mike Cameron-Jones,et al.  Efficient top-down induction of logic programs , 1994, SGAR.

[10]  D. Rubin.  INFERENCE AND MISSING DATA , 1975 .

[11]  Tom M. Mitchell,et al.  Learning to construct knowledge bases from the World Wide Web , 2000, Artif. Intell..

[12]  J. R. Quinlan.  Learning Logical Definitions from Relations , 1990 .

[13]  Rich Caruana,et al.  Multitask Learning: A Knowledge-Based Source of Inductive Bias , 1993, ICML.

[14]  William W. Cohen.  WHIRL: A word-based information representation language , 2000, Artif. Intell..

[15]  David Yarowsky,et al.  Unsupervised Word Sense Disambiguation Rivaling Supervised Methods , 1995, ACL.

[16]  Foster J. Provost,et al.  Handling Missing Values when Applying Classification Models , 2007, J. Mach. Learn. Res..

[17]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[18]  Oren Etzioni,et al.  Open Information Extraction from the Web , 2007, CACM.

[19]  J. Ross Quinlan,et al.  Learning logical definitions from relations , 1990, Machine Learning.