Three ∑P2-complete problems in computational learning theory

The consistency problem associated with a concept classC is to determine, given two setsA andB of examples, whether there exists a conceptc inC such that eachx inA is a positive example ofc and eachy inB is a negative example ofc. We explore in this paper the following intuition: for a concept classC, if the membership problem of determining whether a given example is positive for a concept isNP-complete, then the corresponding consistency problem is likely to be ∑P2-complete. To support this intuition, we prove that the following three consistency problems for concept classes of patterns, graphs and generalized Boolean formulas, whose membership problems are known to beNP-complete, are ∑P2-complete: (a) given two setsA andB of strings, determine whether there exists a patternp such that every string inA is in the languageL(p) and every string inB is not in the languageL(p); (b) given two setsA andB of graphs, determine whether there exists a graphG such that every graph inA is isomorphic to a subgraph ofG and every graph inB is not isomorphic to any subgraph ofG; and (c) given two setsA andB of Boolean formulas, determine whether there exists a 3-CNF Boolean formula θ such that for every ϕ ∈A, θ ∧ ϕ is satisfiable and for every Ψ ∈B, θ ∧ Ψ is not satisfiable. These results suggest that consistendy problems in machine learning are natural candidates for ∑P2-complete problems if the corresponding membership problems are known to beNP-complete.In addition, we prove that the corresponding prediction problems for concept classes of polynomial-time nondeterministic Turing machines, nondeterministic Boolean circuits, generalized Boolean formulas, patterns and graphs are prediction-complete for the classRNP of all concept classes whose membership problems are inNP.

[1]  David Haussler,et al.  Equivalence of models for polynomial learnability , 1988, COLT '88.

[2]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[3]  Klaus W. Wagner,et al.  The complexity of combinatorial problems with succinct input representation , 1986, Acta Informatica.

[4]  David Haussler,et al.  Predicting (0, 1)-functions on randomly drawn points , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[5]  Celia Wrathall,et al.  Complete Sets and the Polynomial-Time Hierarchy , 1976, Theor. Comput. Sci..

[6]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[7]  Robert E. Schapire,et al.  Pattern languages are not learnable , 1990, Annual Conference Computational Learning Theory.

[8]  Albert R. Meyer,et al.  Word problems requiring exponential time(Preliminary Report) , 1973, STOC.

[9]  Ker-I Ko,et al.  A Note on the Two-Variable Pattern-Finding Problem , 1987, J. Comput. Syst. Sci..

[10]  Stephen A. Cook,et al.  The complexity of theorem-proving procedures , 1971, STOC.

[11]  Leonard Pitt,et al.  Prediction-Preserving Reducibility , 1990, J. Comput. Syst. Sci..

[12]  Mihalis Yannakakis,et al.  Equivalences Among Relational Expressions with the Union and Difference Operators , 1980, J. ACM.

[13]  Thiet-Dung Huynh,et al.  Deciding the inequivalence of context-free grammars with 1-letter terminal alphabet is S2p-complete , 1982, 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982).

[14]  Dung T. Huynh.  Deciding the Inequivalence of Context-Free Grammars with 1-Letter Terminal Alphabet is Sigma-p-2-Complete , 1984, Theor. Comput. Sci..

[15]  Richard J. Lipton,et al.  Some connections between nonuniform and uniform complexity classes , 1980, STOC '80.

[16]  Albert R. Meyer,et al.  The Equivalence Problem for Regular Expressions with Squaring Requires Exponential Space , 1972, SWAT.

[17]  Larry J. Stockmeyer,et al.  The Polynomial-Time Hierarchy , 1976, Theor. Comput. Sci..

[18]  Leonard Pitt,et al.  A polynomial-time algorithm for learning k-variable pattern languages from examples , 1989, COLT '89.

[19]  Wen-Guey Tzeng,et al.  Learning String Patterns and Tree Patterns from Examples , 1990, ML.

[20]  Leonard Pitt,et al.  On the necessity of Occam algorithms , 1990, STOC '90.

[21]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[22]  Frank Harary,et al.  Graph Theory , 2016 .

[23]  Dana Angluin,et al.  Finding Patterns Common to a Set of Strings , 1980, J. Comput. Syst. Sci..

[24]  D. Angluin.  Queries and Concept Learning , 1988 .

[25]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[26]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .