Computational Learning Theory and Language Acquisition

Computational learning theory explores the limits of learnability. Studying language acquisition from this perspective involves identifying classes of languages that are learnable from the available data, within the limits of time and computational resources available to the learner. Different models of learning can yield radically different learnability results, where these depend on the assumptions of the model about the nature of the learning process, and the data, time, and resources that learners have access to. To the extent that such assumptions accurately reflect human language learning, a model that invokes them can offer important insights into the formal properties of natural languages, and the way in which their representations might be efficiently acquired. In this chapter we consider several computational learning models that have been applied to the language learning task. Some of these have yielded results that suggest that the class of natural languages cannot be efficiently learned from the primary linguistic data (PLD) available to children, through

[1]  John A. Goldsmith,et al.  Unsupervised Learning of the Morphology of a Natural Language , 2001, CL.

[2]  Shalom Lappin,et al.  Unsupervised Learning and Grammar Induction , 2010 .

[3]  Alexander Clark,et al.  Combining Distributional and Morphological Information for Part of Speech Induction , 2003, EACL.

[4]  Glenn Carroll,et al.  Two Experiments on Learning Probabilistic Dependency Grammars from Corpora , 1992 .

[5]  David Haussler,et al.  Equivalence of models for polynomial learnability , 1988, COLT '88.

[6]  S. Pinker,et al.  The faculty of language: what's special about it? , 2005, Cognition.

[7]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[8]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[9]  Yehoshua Bar-Hillel,et al.  On syntactical categories , 1950, Journal of Symbolic Logic.

[10]  S. Anderson The Logical Structure of Linguistic Theory , 2009 .

[11]  Sally A. Goldman,et al.  Teaching a Smarter Learner , 1996, J. Comput. Syst. Sci..

[12]  Michael Collins,et al.  Head-Driven Statistical Models for Natural Language Parsing , 2003, CL.

[13]  Robert A. Wilson,et al.  Book Reviews: The MIT Encyclopedia of the Cognitive Sciences , 2000, CL.

[14]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.

[15]  Zellig S. Harris,et al.  Distributional Structure , 1954 .

[16]  J. Lambek The Mathematics of Sentence Structure , 1958 .

[17]  Shalom Lappin,et al.  Linguistic Nativism and the Poverty of the Stimulus , 2011 .

[18]  Shuly Wintner Formal Language Theory , 2010 .

[19]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.

[20]  Dana Angluin,et al.  Inference of Reversible Languages , 1982, JACM.

[21]  Steven Pinker,et al.  Language learnability and language development , 1985 .

[22]  Denise Brandão de Oliveira e Britto,et al.  The faculty of language , 2007 .

[23]  Hinrich Schütze,et al.  Distributional Part-of-Speech Tagging , 1995, EACL.

[24]  Alexander Clark,et al.  A Learnable Representation for Syntax Using Residuated Lattices , 2009, FG.

[25]  Rens Bod,et al.  A Linguistic Investigation into Unsupervised DOP , 2007 .

[26]  Rens Bod,et al.  An All-Subtrees Approach to Unsupervised Parsing , 2006, ACL.

[27]  José Oncina,et al.  Learning deterministic regular grammars from stochastic samples in polynomial time , 1999, RAIRO Theor. Informatics Appl..

[28]  Steven Abney,et al.  Statistical Methods and Linguistics , 2002 .

[29]  Leslie G. Valiant,et al.  Cryptographic limitations on learning Boolean formulae and finite automata , 1994, JACM.

[30]  Marek A. Przezdziecki Vowel harmony and coarticulation in three dialects of Yoruba : phonetics determining phonology , 2005 .

[31]  Stephen Crain,et al.  Why language acquisition is a snap , 2002 .

[32]  Alexander Clark,et al.  Efficient, Correct, Unsupervised Learning for Context-Sensitive Languages , 2010, CoNLL.

[33]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[34]  C. Bishop The MIT Encyclopedia of the Cognitive Sciences , 1999 .

[35]  Noam Chomsky Three Factors in Language Design , 2005, Linguistic Inquiry.

[36]  Chris Fox,et al.  The Handbook of Computational Linguistics and Natural Language Processing , 2010 .

[37]  Barbara C. Scholz,et al.  Empirical assessment of stimulus poverty arguments , 2002 .

[38]  Rens Bod,et al.  Is the End of Supervised Parsing in Sight? , 2007, ACL.

[39]  Kent Johnson Gold’s Theorem and Cognitive Science* , 2004, Philosophy of Science.

[40]  Noam Chomsky,et al.  The Minimalist Program , 1992 .

[41]  Noam Chomsky,et al.  New horizons in the study of language , 1997 .

[42]  Rens Bod,et al.  From Exemplar to Grammar: A Probabilistic Analogy-Based Model of Language Learning , 2009, Cogn. Sci..

[43]  J. Myhill Review: Yehoshua Bar-Hillel, On Syntactical Categories , 1950 .

[44]  Noam Chomsky,et al.  Lectures on Government and Binding , 1981 .

[45]  Stuart M. Shieber,et al.  Evidence against the context-freeness of natural language , 1985 .

[46]  Robert J. Matthews,et al.  The Plausibility of Rationalism , 1984 .

[47]  Norbert Hornstein,et al.  Explanation in Linguistics: The Logical Problem of Language Acquisition , 1982 .

[48]  B. MacWhinney The CHILDES project: tools for analyzing talk , 1992 .

[49]  Charles D. Yang,et al.  Knowledge and learning in natural language , 2000 .

[50]  Paul M. B. Vitányi,et al.  ‘Ideal learning’ of natural language: Positive results about learning from positive evidence , 2007 .

[51]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[52]  Dan Klein,et al.  Corpus-Based Induction of Syntactic Structure: Models of Dependency and Constituency , 2004, ACL.

[53]  Daniel Jurafsky,et al.  Knowledge-Free Induction of Inflectional Morphologies , 2001, NAACL.

[54]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[55]  James Jay Horning,et al.  A study of grammatical inference , 1969 .

[56]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[57]  Fernando C Pereira Formal grammar and information theory: together again? , 2000, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[58]  Bernhard Ganter,et al.  Formal Concept Analysis: Mathematical Foundations , 1998 .

[59]  M A Nowak,et al.  Evolution of universal grammar. , 2001, Science.

[60]  Maria T. Pazienza,et al.  Information Extraction , 2002, Lecture Notes in Computer Science.

[61]  Iris van Rooij,et al.  The Tractable Cognition Thesis , 2008, Cogn. Sci..

[62]  Alexander Clark,et al.  Polynomial Identification in the Limit of Substitutable Context-free Languages , 2005 .

[63]  Naoki Abe,et al.  On the computational complexity of approximating distributions by probabilistic automata , 1990, Machine Learning.

[64]  Janet D. Fodor,et al.  Understanding stimulus poverty arguments , 2002 .

[65]  P. Niyogi,et al.  A language learning model for finite parameter spaces , 1996, Cognition.

[66]  SHALOM LAPPIN,et al.  Machine learning theory and practice as a source of insight into universal grammar , 2007 .

[67]  A. Grafstein MIT Encyclopedia of the Cognitive Sciences , 2000 .

[68]  Eugene Charniak,et al.  Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking , 2005, ACL.

[69]  Robert C. Berwick,et al.  Treebank parsing and knowledge of language: a cognitive perspective , 2009 .

[70]  Charles D. Yang The great number crunch1 , 2008, Journal of Linguistics.

[71]  Jimmy J. Lin,et al.  Evaluation of NLP Systems , 2010 .

[72]  S. Curtiss Genie: A Psycholinguistic Study of a Modern-Day "Wild Child" , 1977 .

[73]  E. Mark Gold,et al.  Complexity of Automaton Identification from Given Data , 1978, Inf. Control..

[74]  John Myhill,et al.  Bar-Hillel Yehoshua. On syntactical categories. The journal of symbolic logic, vol. 15 (1950), pp. 1–16. , 1950, Journal of Symbolic Logic.

[75]  Dana Angluin,et al.  When won't membership queries help? , 1991, STOC '91.

[76]  S. Laurence,et al.  The Poverty of the Stimulus Argument , 2001, The British Journal for the Philosophy of Science.

[77]  Noam Chomsky,et al.  Poverty of the Stimulus' Revisited: Recent Challenges Reconsidered , 2008 .