Learning implicitly in reasoning in PAC-Semantics

We consider the problem of answering queries about formulas of propositional logic based on background knowledge partially represented explicitly as other formulas, and partially represented as partially obscured examples independently drawn from a fixed probability distribution, where the queries are answered with respect to a weaker semantics than usual -- PAC-Semantics, introduced by Valiant (2000) -- that is defined using the distribution of examples. We describe a fairly general, efficient reduction to limited versions of the decision problem for a proof system (e.g., bounded space treelike resolution, bounded degree polynomial calculus, etc.) from corresponding versions of the reasoning problem where some of the background knowledge is not explicitly given as formulas, only learnable from the examples. Crucially, we do not generate an explicit representation of the knowledge extracted from the examples, and so the "learning" of the background knowledge is only done implicitly. As a consequence, this approach can utilize formulas as background knowledge that are not perfectly valid over the distribution---essentially the analogue of agnostic learning here.

[1]  Leslie G. Valiant,et al.  A First Experimental Demonstration of Massive Knowledge Infusion , 2008, KR.

[2]  Loizos Michael.  Partial observability and learnability , 2010, Artif. Intell..

[3]  B. Buchberger,et al.  Grobner Bases : An Algorithmic Method in Polynomial Ideal Theory , 1985 .

[4]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[5]  Oliver Kullmann,et al.  Investigating a general hierarchy of polynomially decidable classes of CNF's based on short tree-like resolution proofs , 1999, Electron. Colloquium Comput. Complex..

[6]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[7]  John McCarthy,et al.  Applications of Circumscription to Formalizing Common Sense Knowledge , 1987, NMR.

[8]  Moni Naor,et al.  Search problems in the decision tree model , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[9]  R. Schapire,et al.  Toward Efficient Agnostic Learning , 1994 .

[10]  R. Horton.  EROSIONAL DEVELOPMENT OF STREAMS AND THEIR DRAINAGE BASINS; HYDROPHYSICAL APPROACH TO QUANTITATIVE MORPHOLOGY , 1945 .

[11]  David Chapman,et al.  Planning for Conjunctive Goals , 1987, Artif. Intell..

[12]  J. Krajícek.  On the weak pigeonhole principle , 2001 .

[13]  Raymond Reiter,et al.  The Frame Problem in the Situation Calculus: A Simple Solution (Sometimes) and a Completeness Result for Goal Regression , 1991, Artificial and Mathematical Theory of Computation.

[14]  J. Kraj.  On the Weak Pigeonhole Principle , 2001 .

[15]  Moisés Goldszmidt,et al.  Action Networks: A Framework for Reasoning about Actions and Change under Uncertainty , 1994, UAI.

[16]  A. N. Strahler.  Hypsometric (area-altitude) analysis of erosional topography. , 1952 .

[17]  John McCarthy,et al.  Circumscription - A Form of Non-Monotonic Reasoning , 1980, Artif. Intell..

[18]  Raymond Reiter,et al.  A Logic for Default Reasoning , 1987, Artif. Intell..

[19]  Roberto J. Bayardo,et al.  Using CSP Look-Back Techniques to Solve Real-World SAT Instances , 1997, AAAI/IAAI.

[20]  Michael Alekhnovich,et al.  Space Complexity in Propositional Calculus , 2002, SIAM J. Comput..

[21]  Henry A. Kautz,et al.  Towards Understanding and Harnessing the Potential of Clause Learning , 2004, J. Artif. Intell. Res..

[22]  Drew McDermott,et al.  Nonmonotonic Logic and Temporal Projection , 1987, Artif. Intell..

[23]  Dan Roth,et al.  Learning to Reason with a Restricted View , 1995, COLT '95.

[24]  Bart Selman,et al.  Pushing the Envelope: Planning, Propositional Logic and Stochastic Search , 1996, AAAI/IAAI, Vol. 2.

[25]  Donald W. Loveland,et al.  A machine program for theorem-proving , 2011, CACM.

[26]  Bart Selman,et al.  Planning as Satisfiability , 1992, ECAI.

[27]  Jacobo Torán,et al.  Space Bounds for Resolution , 1999, STACS.

[28]  Leslie G. Valiant,et al.  Robust logics , 1999, STOC '99.

[29]  Dan Roth,et al.  Learning to reason , 1994, JACM.

[30]  Avi Wigderson,et al.  Restriction access , 2012, ITCS '12.

[31]  Sharad Malik,et al.  Efficient conflict driven learning in a Boolean satisfiability solver , 2001, IEEE/ACM International Conference on Computer Aided Design. ICCAD 2001. IEEE/ACM Digest of Technical Papers (Cat. No.01CH37281).

[32]  Keiji Kanazawa,et al.  A model for reasoning about persistence and causation , 1989 .

[33]  John McCarthy,et al.  SOME PHILOSOPHICAL PROBLEMS FROM THE STANDPOINT OF ARTI CIAL INTELLIGENCE , 1987 .

[34]  David Haussler,et al.  Learning decision trees from random examples , 1988, COLT '88.

[35]  Nick Littlestone,et al.  Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm , 2004, Machine Learning.

[36]  Vasek Chvátal,et al.  Edmonds polytopes and a hierarchy of combinatorial problems , 1973, Discret. Math..

[37]  Zvi Galil.  On Resolution with Clauses of Bounded Size , 1977, SIAM J. Comput..

[38]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[39]  Johannes Klaus Fichte,et al.  Clause-Learning Algorithms with Many Restarts and Bounded-Width Resolution , 2011, J. Artif. Intell. Res..

[40]  Toniann Pitassi,et al.  Simplified and improved resolution lower bounds , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[41]  Luc De Raedt,et al.  Basic Principles of Learning Bayesian Logic Programs , 2008, Probabilistic Inductive Logic Programming.

[42]  Adnan Darwiche,et al.  On the power of clause-learning SAT solvers as resolution engines , 2011, Artif. Intell..

[43]  Bart Selman,et al.  Heavy-Tailed Distributions in Combinatorial Search , 1997, CP.

[44]  J. P. Marques,et al.  GRASP : A Search Algorithm for Propositional Satisfiability , 1999 .

[45]  Maria Luisa Bonet,et al.  Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (2008) , 2022 .

[46]  Hilary Putnam,et al.  A Computing Procedure for Quantification Theory , 1960, JACM.

[47]  Russell Impagliazzo,et al.  Using the Groebner basis algorithm to find proofs of unsatisfiability , 1996, STOC '96.

[48]  Eli Ben-Sasson,et al.  Short proofs are narrow—resolution made simple , 2001, JACM.

[49]  William J. Cook,et al.  On the complexity of cutting-plane proofs , 1987, Discret. Appl. Math..

[50]  Ralph E. Gomory,et al.  Solving linear programming problems in integers , 1960 .