Theory revision with queries: Horn, read-once, and parity formulas

A theory, in this context, is a Boolean formula; it is used to classify instances, or truth assignments. Theories can model real-world phenomena, and can do so more or less correctly. The theory revision, or concept revision, problem is to correct a given, roughly correct concept. This problem is considered here in the model of learning with equivalence and membership queries. A revision algorithm is considered efficient if the number of queries it makes is polynomial in the revision distance between the initial theory and the target theory, and polylogarithmic in the number of variables and the size of the initial theory. The revision distance is the minimal number of syntactic revision operations, such as the deletion or addition of literals, needed to obtain the target theory from the initial theory. Efficient revision algorithms are given for Horn formulas and read-once formulas, where revision operators are restricted to deletions of variables or clauses, and for parity formulas, where revision operators include both deletions and additions of variables. We also show that the query complexity of the read-once revision algorithm is near-optimal.

[1]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[2]  Derek H. Sleeman,et al.  Effective and Efficient Knowledge Base Refinement , 1999, Machine Learning.

[3]  Luc De Raedt,et al.  Logical Settings for Concept-Learning , 1997, Artif. Intell..

[4]  Philip M. Long,et al.  Structural Results About On-line Learning Models With and Without Queries , 1999, Machine Learning.

[5]  Hans K. Buning,et al.  Propositional Logic: Deduction and Algorithms , 1999 .

[6]  Robert A. Kowalski,et al.  The Semantics of Predicate Logic as a Programming Language , 1976, JACM.

[7]  J. Taylor,et al.  Switching and finite automata theory, 2nd ed. , 1980, Proceedings of the IEEE.

[8]  Daniele Mundici.  Functions Computed by Monotone Boolean Formulas with no Repeated Variables , 1989, Theor. Comput. Sci..

[9]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[10]  Jean H. Gallier,et al.  Linear-Time Algorithms for Testing the Satisfiability of Propositional Horn Formulae , 1984, J. Log. Program..

[11]  Michael Frazier,et al.  Learning conjunctions of Horn clauses , 2004, Machine Learning.

[12]  Judy Goldsmith,et al.  More theory revision with queries (extended abstract) , 2000, STOC '00.

[13]  Michael Frazier.  Matters horn and other features in the computational learning theory landscape: the notion of membership , 1994, COLT 1994.

[14]  Michael R. Lowry,et al.  Learning Physical Descriptions From Functional Definitions, Examples, and Precedents , 1983, AAAI.

[15]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[16]  Avrim Blum,et al.  Fast learning of k-term DNF formulas with queries , 1992, STOC '92.

[17]  Nader H. Bshouty,et al.  Asking Questions to Minimize Errors , 1996, J. Comput. Syst. Sci..

[18]  Judy Goldsmith,et al.  Theory Revision with Queries: DNF Formulas , 2002, Machine Learning.

[19]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[20]  Russell Greiner.  The Complexity of Theory Revision , 1995, IJCAI.

[21]  Ronen Feldman,et al.  Bias-Driven Revision of Logical Domain Theories , 1993, J. Artif. Intell. Res..

[22]  Lisa Hellerstein,et al.  Read-thrice DNF is hard to learn with membership and equivalence queries , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[23]  S. Wrobel.  Concept Formation and Knowledge Revision , 1994, Springer US.

[24]  Leslie G. Valiant,et al.  Projection Learning , 1998, COLT' 98.

[25]  Raymond J. Mooney,et al.  Theory Refinement Combining Analytical and Empirical Methods , 1994, Artif. Intell..

[26]  Rina Dechter,et al.  Structure Identification in Relational Data , 1992, Artif. Intell..

[27]  Balázs Szörényi,et al.  Improved Algorithms for Theory Revision with Queries , 2000, COLT.

[28]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[29]  S. Wrobel.  First Order Theory Reenement , 1996 .

[30]  Peter Damaschke.  Adaptive versus nonadaptive attribute-efficient learning , 1998, STOC '98.

[31]  Edward A. Feigenbaum,et al.  Switching and Finite Automata Theory: Computer Science Series , 1990 .

[32]  Judy Goldsmith,et al.  More theory revision with queries , 2000, STOC 2000.

[33]  Silvio Micali,et al.  How to construct random functions , 1986, JACM.

[34]  Ingo Wegener,et al.  The complexity of Boolean functions , 1987 .

[35]  Dana Angluin,et al.  When won't membership queries help? , 1991, STOC '91.

[36]  Shlomo Argamon,et al.  Tractability of Theory Patching , 1998, J. Artif. Intell. Res..

[37]  Lisa Hellerstein,et al.  Complexity theoretic hardness results for query learning , 1998, computational complexity.

[38]  Samson Abramsky,et al.  Handbook of logic in computer science (vol. 1): background: mathematical structures , 1993 .

[39]  Lisa Hellerstein,et al.  PAC learning with irrelevant attributes , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[40]  R. Mooney.  A Preliminary PAC Analysis of Theory Revision , 1995 .

[41]  Vijay Raghavan,et al.  Read-Twice DNF Formulas are Properly Learnable , 1994, Inf. Comput..

[42]  Lisa Hellerstein,et al.  Attribute-efficient learning in query and mistake-bound models , 1996, COLT '96.

[43]  Nader H. Bshouty,et al.  On the exact learning of formulas in parallel , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[44]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.

[45]  Nader H. Bshouty.  Exact Learning Boolean Function via the Monotone Theory , 1995, Inf. Comput..

[46]  Russell Greiner.  The Complexity of Revising Logic Programs , 1999, J. Log. Program..

[47]  Dana Angluin.  Negative results for equivalence queries , 1990, Mach. Learn..

[48]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[49]  György Turán,et al.  On theory revision with queries , 1999, COLT '99.

[50]  Wolfgang Maass,et al.  Lower Bound Methods and Separation Results for On-Line Learning Models , 2004, Machine Learning.

[51]  J. A. Makowsky.  Model theory and computer science: an appetizer , 1993, LICS 1993.

[52]  Marek Karpinski,et al.  Learning read-once formulas with queries , 1993, JACM.

[53]  Alfred Horn,et al.  On sentences which are true of direct unions of algebras , 1951, Journal of Symbolic Logic.

[54]  Ronen Feldman.  Probabilistic Revision of Logical Doman Theories , 1993 .

[55]  Michael E. Saks,et al.  Combinatorial characterization of read-once formulae , 1993, Discret. Math..

[56]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[57]  J. C. C. McKinsey,et al.  The decision problem for some classes of sentences without quantifiers , 1943, Journal of Symbolic Logic.

[58]  Michael Frazier,et al.  Learning From Entailment: An Application to Propositional Horn Sentences , 1993, ICML.

[59]  N. Littlestone,et al.  Learning in the presence of finitely or infinitely many irrelevant attributes , 1991, COLT '91.