The Complexity of Theory Revision

A knowledge-based system uses its database (a k a its "theory") to produce answers to the queries it receives. Unfortunately, these answers may be incorrect if the underlying theory is faulty Standard "theory revision" systems use a given set of "labeled queries" (each a query paired with its correct answer) to transform the given theory, by adding and/or deleting either rules and/or antecedents, into a related theory that is as accurate as possible. After formally defining the theory revision task and bounding its sample complexity, this paper addresses the task's computational complexity. It first proves that, unless P = NP, no polynomial time algorithm can identify the optimal theory, even given the exact distribution of queries, except in the most trivial of situations. It also shows that, except in such trivial situations, no polynomial-time algorithm can produce a theory whose inaccuracy is even close (i e, within a particular polynomial factor) to optimal. These results justify the standard practice of hill-climbing to a locally-optimal theory, based on a given set of labeled samples.

[1]  Béla Bollobás,et al.  Random Graphs , 1985 .

[2]  G. Plotkin.  Automatic Methods of Inductive Inference , 1972 .

[3]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[4]  Raymond J. Mooney,et al.  Symbolic Revision of Theories with M-of-N Rules , 1993, IJCAI.

[5]  Bart Selman,et al.  Knowledge Compilation using Horn Approximations , 1991, AAAI.

[6]  Dan Roth,et al.  Reasoning with Models , 1994, Artif. Intell..

[7]  Russell Greiner.  The Challenge of Revising an Impure Theory , 1995, ICML.

[8]  William W. Cohen.  Pac-Learning Recursive Logic Programs: Efficient Algorithms , 1994, J. Artif. Intell. Res..

[9]  R. Schapire.  Toward Eecient Agnostic Learning , 1992 .

[10]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[11]  Judea Pearl,et al.  On the Logic of Iterated Belief Revision , 1994, Artif. Intell..

[12]  Ehud Shapiro,et al.  Algorithmic Program Debugging , 1983 .

[13]  William W. Cohen.  Pac-Learning Non-Recursive Prolog Clauses , 1995, Artif. Intell..

[14]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[15]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[16]  Hirofumi Katsuno,et al.  On the Difference between Updating a Knowledge Base and Revising It , 1991, KR.

[17]  Bart Selman,et al.  Horn Approximations of Empirical Data , 1995, Artif. Intell..

[18]  Jack Minker,et al.  Logic and Data Bases , 1978, Springer US.

[19]  Dan Roth,et al.  Learning to reason , 1994, JACM.

[20]  Christos H. Papadimitriou,et al.  Incremental Recompilation of Knowledge , 1994, AAAI.

[21]  Michael Frazier,et al.  Learning From Entailment: An Application to Propositional Horn Sentences , 1993, ICML.

[22]  Jon Doyle,et al.  Two Theses of Knowledge Representation: Language Restrictions, Taxonomic Classification, and the Utility of Representation Services , 1991, Artif. Intell..

[23]  Saso Dzeroski,et al.  PAC-learnability of determinate logic programs , 1992, COLT '92.

[24]  P G rdenfors,et al.  Knowledge in flux: modeling the dynamics of epistemic states , 1988 .

[25]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[26]  David C. Wilkins,et al.  The Refinement of Probabilistic Rule Sets: Sociopathic Interactions , 1994, Artif. Intell..

[27]  William W. Cohen.  Abductive explanation-based learning: A solution to the multiple inconsistent explanation problem , 1992, Machine Learning.

[28]  Marco Valtorta,et al.  Refinement of Uncertain Rule Bases via Reduction , 1995, Int. J. Approx. Reason..

[29]  Rina Dechter,et al.  Structure Identification in Relational Data , 1992, Artif. Intell..

[30]  Alexander Borgida,et al.  Hierarchical Knowledge Bases and Efficient Disjunctive Reasoning , 1989, KR.

[31]  Christos H. Papadimitriou,et al.  Incremental recompilation of knowledge (extended abstract) , 1994, AAAI 1994.

[32]  Linda Sellie,et al.  Toward efficient agnostic learning , 1992, COLT '92.

[33]  Stephen Muggleton,et al.  Machine Invention of First Order Predicates by Inverting Resolution , 1988, ML.

[34]  Bart Selman,et al.  Reasoning With Characteristic Models , 1993, AAAI.

[35]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[36]  Catriel Beeri,et al.  Optimizing existential datalog queries , 1988, PODS.

[37]  Peter G. Politakis,et al.  Empirical analysis for expert systems , 1985 .

[38]  Michael J. Pazzani,et al.  A Methodology for Evaluating Theory Revision Systems: Results with Audrey II , 1993, IJCAI.

[39]  Peter Gärdenfors,et al.  On the logic of theory change: Partial meet contraction and revision functions , 1985, Journal of Symbolic Logic.

[40]  R. Mooney.  A Preliminary PAC Analysis of Theory Revision , 1995 .

[41]  Alexander Kogan,et al.  Knowing what doesn't Matter: Exploiting the Omission of Irrelevant Data , 1997, Artif. Intell..

[42]  Raymond J. Mooney,et al.  Theory Refinement Combining Analytical and Empirical Methods , 1994, Artif. Intell..

[43]  Mukesh Dalal,et al.  Investigations into a Theory of Knowledge Base Revision , 1988, AAAI.

[44]  Dale Schuurmans,et al.  Learning Useful Horn Approximations , 1992, KR.

[45]  Alessandro Panconesi,et al.  Completeness in Approximation Classes , 1989, Inf. Comput..

[46]  Craig Boutilier,et al.  Revision Sequences and Nested Conditionals , 1993, IJCAI.

[47]  Carsten Lund,et al.  On the hardness of approximating minimization problems , 1993, STOC.

[48]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[49]  Frederick Hayes-Roth,et al.  The state of knowledge-based systems , 1994, CACM.

[50]  Geoffrey G. Towell,et al.  Symbolic knowledge and neural networks: insertion, refinement and extraction , 1992 .

[51]  Robert E. Schapire,et al.  Efficient distribution-free learning of probabilistic concepts , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[52]  Keith L. Clark,et al.  Negation as Failure , 1987, Logic and Data Bases.

[53]  Marco Valtorta.  Some results on the computational complexity of refining confidence factors , 1991, Int. J. Approx. Reason..

[54]  William W. Cohen.  Pac-learning Recursive Logic Programs: Negative Results , 1994, J. Artif. Intell. Res..

[55]  William W. Cohen.  Learning from Textbook Knowledge: A Case Study , 1990, AAAI.

[56]  Nathan Linial,et al.  Results on learnability and the Vapnik-Chervonenkis dimension , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[57]  Derek H. Sleeman,et al.  Automating the Refinement of Knowledge-Based Systems , 1990, ECAI.

[58]  Hector J. Levesque,et al.  Foundations of a Functional Approach to Knowledge Representation , 1984, Artif. Intell..

[59]  Georg Gottlob,et al.  On the Complexity of Propositional Knowledge Base Revision, Updates, and Counterfactuals , 1992, Artif. Intell..