Quantifying Prior Determination Knowledge Using the PAC Learning Model

Prior knowledge, or bias, regarding a concept can reduce the number of examples needed to learn it. Probably Approximately Correct (PAC) learning is a mathematical model of concept learning that can be used to quantify the reduction in the number of examples due to different forms of bias. Thus far, PAC learning has mostly been used to analyzesyntactic bias, such as limiting concepts to conjunctions of boolean prepositions. This paper demonstrates that PAC learning can also be used to analyzesemantic bias, such as a domain theory about the concept being learned. The key idea is to view the hypothesis space in PAC learning as that consistent withall prior knowledge, syntactic and semantic. In particular, the paper presents an analysis ofdeterminations, a type of relevance knowledge. The results of the analysis reveal crisp distinctions and relations among different determinations, and illustrate the usefulness of an analysis based on the PAC learning model.

[1]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[2]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[3]  Richard E. Korf,et al.  Macro-Operators: A Weak Method for Learning , 1985, Artif. Intell..

[4]  Robert J. Hall.  Learning by Failing to Explain , 1986, AAAI.

[5]  Balas K. Natarajan,et al.  On learning Boolean functions , 1987, STOC.

[6]  Stuart J. Russell,et al.  A Logical Approach to Reasoning by Analogy , 1987, IJCAI.

[7]  Stuart J. Russell.  Analogy and Single-Instance Generalization , 1987 .

[8]  Stuart J. Russell.  Analogical and inductive reasoning , 1987 .

[9]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[10]  Todd R. Davies,et al.  Determination, Uniformity, and Relevance: Normative Criteria for Generalization and Reasoning by Analogy , 1988 .

[11]  Prasad Tadepalli,et al.  Two New Frameworks for Learning , 1988, ML.

[12]  Sridhar Mahadevan,et al.  On the Tractability of Learning from Incomplete Theories , 1988, ML.

[13]  David Haussler,et al.  Predicting {0,1}-functions on randomly drawn points , 1988, COLT '88.

[14]  Stuart J. Russell.  Tree-Structured Bias , 1988, AAAI.

[15]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[16]  Sridhar Mahadevan.  Using Determinations in EBL: A Solution to the incomplete Theory Problem , 1989, ML.

[17]  Andrea Pohoreckyj Danyluk,et al.  Finding New Rules for Incomplete Theories: Explicit Biases for Induction with Contextual Information , 1989, ML.

[18]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[19]  Sturart J. Russell,et al.  The use of knowledge in analogy and induction , 1989 .

[20]  Thomas G. Dietterich.  Limitations on Inductive Learning , 1989, ML.

[21]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[22]  Benjamin N. Grosof,et al.  A Sketch of Autonomous Learning using Declarative Bias , 1990 .

[23]  Leonard Pitt,et al.  Prediction-Preserving Reducibility , 1990, J. Comput. Syst. Sci..

[24]  H. Hirsh.  Incremental Version-Space Merging: A General Framework for Concept Learning , 1990 .

[25]  Prasad Tadepalli,et al.  A Formalization of Explanation-Based Macro-operator Learning , 1991, IJCAI.

[26]  Marie desJardins.  Probabilistic Evaluating of Bias for Learning Systems , 1991, ML.

[27]  Rodney A. Brooks,et al.  Intelligence Without Reason , 1991, IJCAI.

[28]  Translator-IEEE Expert staff.  Machine Learning: A Theoretical Approach , 1992, IEEE Expert.

[29]  Tom Bylander,et al.  Complexity Results for Serial Decomposability , 1992, AAAI.

[30]  Prasad Tadepalli,et al.  Learning from Queries and Examples with Tree-structured Bias , 1993, ICML.

[31]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.

[32]  Leslie G. Valiant,et al.  Cryptographic limitations on learning Boolean formulae and finite automata , 1994, JACM.

[33]  David B. Leake.  Artiicial Intelligence , 2001 .

[34]  Balas K. Natarajan,et al.  On learning sets and functions , 2004, Machine Learning.

[35]  Tom M. Mitchell,et al.  Explanation-Based Generalization: A Unifying View , 1986, Machine Learning.

[36]  Ronald L. Rivest,et al.  Learning decision lists , 2004, Machine Learning.

[37]  R. Mooney,et al.  Explanation-Based Learning: An Alternative View , 1986, Machine Learning.

[38]  M. Pazzani,et al.  The Utility of Knowledge in Inductive Learning , 1992, Machine Learning.