Learning Recursion: Multiple Nested and Crossed Dependencies

Language acquisition in both natural and artificial language learning settings crucially depends on extracting information from ordered sequences. A shared sequence learning mechanism is thus assumed to underlie both natural and artificial language learning. A growing body of empirical evidence is consistent with this hypothesis. By means of artificial language learning experiments, we may therefore gain more insight in this shared mechanism. In this paper, we review empirical evidence from artificial language learning and computational modeling studies, as well as natural language data, and suggest that there are two key factors that help determine processing complexity in sequence learning, and thus in natural language processing. We propose that the specific ordering of non-adjacent dependencies (i.e. nested or crossed), as well as the number of non-adjacent dependencies to be resolved simultaneously (i.e. two or three) are important factors in gaining more insight into the boundaries of human sequence learning; and thus, also in natural language processing. The implications for theories of linguistic competence are discussed.

[1]  George A. Miller,et al.  Free Recall of Self-Embedded English Sentences , 1964, Inf. Control..

[2]  Jacques Mehler,et al.  Do Humans Really Learn AnBn Artificial Grammars From Exemplars? , 2008, Cogn. Sci..

[3]  Donald J. Foss,et al.  Some effects of memory limitation upon sentence comprehension and recall , 1970 .

[4]  Paul Hagstrom,et al.  Differences Between Korean and Japanese Processing Overload , 1997 .

[5]  Noam Chomsky,et al.  The faculty of language: what is it, who has it, and how did it evolve? , 2002, Science.

[6]  Guillén Fernández,et al.  Neural correlates of artificial syntactic structure classification , 2006, NeuroImage.

[7]  Peter Hagoort,et al.  Implicit Learning and Dyslexia , 2008, Annals of the New York Academy of Sciences.

[8]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[9]  Pienie Zwitserlood,et al.  Syntactic structure and artificial grammar learning: The learnability of embedded hierarchical structures , 2008, Cognition.

[10]  Morten H. Christiansen,et al.  Sequential Expectations: The Role of Prediction-Based Learning in Language , 2010, Top. Cogn. Sci..

[11]  J. S. Evans,et al.  Understanding sentences with relative clauses , 1976, Memory & cognition.

[12]  M. Braine,et al.  Short-term memory limitations on decoding self-embedded sentences , 1974 .

[13]  Fred Karlsson,et al.  Constraints on multiple center-embedding of clauses , 2007 .

[14]  Timothy Q. Gentner,et al.  Recursive syntactic pattern learning by songbirds , 2006, Nature.

[15]  John Payne Pullum, Geoffrey K. , 2012 .

[16]  D. Everett Cultural Constraints on Grammar and Cognition in Pirahã , 2005, Current Anthropology.

[17]  Michael C. Corballis,et al.  Recursion, Language, and Starlings , 2007, Cogn. Sci..

[18]  Elissa L. Newport,et al.  Statistical Learning of Syntax: The Role of Transitional Probability , 2007 .

[19]  I. M. Schlesinger,et al.  WHY A SENTENCE IN WHICH A SENTENCE IN WHICH A SENTENCE IS EMBEDDED IS EMBEDDED IS DIFFICULT , 1975 .

[20]  D. J. Foss,et al.  Decision processes during sentence comprehension: Effects of surface structure reconsidered , 1970 .

[21]  J. Saffran The Use of Predictive Dependencies in Language Learning , 2001 .

[22]  J. Deese,et al.  Comprehensibility and Subject-Verb Relations in Complex Sentences. , 1971 .

[23]  R. Gómez Variability and Detection of Invariant Structure , 2002, Psychological science.

[24]  Alaa A. Kharbouch,et al.  Three models for the description of language , 1956, IRE Trans. Inf. Theory.

[25]  Karl Magnus Petersson,et al.  On the relevance of the neurobiological analogue of the finite-state architecture , 2005, Neurocomputing.

[26]  A. Rey,et al.  Does the mastery of center-embedded linguistic structures distinguish humans from nonhuman primates? , 2005, Psychonomic bulletin & review.

[27]  P. Greenfield,et al.  Language, tools and brain: The ontogeny and phylogeny of hierarchically organized sequential behavior , 1991, Behavioral and Brain Sciences.

[28]  A. Reber Implicit learning and tacit knowledge , 1993 .

[29]  Richard N Aslin,et al.  Statistical learning of adjacent and nonadjacent dependencies among nonlinguistic sounds , 2009, Psychonomic bulletin & review.

[30]  Morten H. Christiansen,et al.  Modality-constrained statistical learning of tactile, visual, and auditory sequences. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[31]  Elizabeth K. Johnson,et al.  Statistical learning of tone sequences by human infants and adults , 1999, Cognition.

[32]  Hava T. Siegelmann,et al.  Neural networks and analog computation - beyond the Turing limit , 1999, Progress in theoretical computer science.

[33]  P. Perruchet,et al.  Implicit learning and statistical learning: one phenomenon, two approaches , 2006, Trends in Cognitive Sciences.

[34]  Sarah C. Creel,et al.  Distant melodies: statistical learning of nonadjacent dependencies in tone sequences. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[35]  Axel Cleeremans,et al.  Implicit learning: news from the front , 1998, Trends in Cognitive Sciences.

[36]  Alfred Anwander,et al.  Segregating the core computational faculty of human language from working memory , 2009, Proceedings of the National Academy of Sciences.

[37]  E. Newport,et al.  Learning at a distance I. Statistical learning of non-adjacent dependencies , 2004, Cognitive Psychology.

[38]  Jutta L. Mueller,et al.  Learnability of Embedded Syntactic Structures Depends on Prosodic Cues , 2010, Cogn. Sci..

[39]  David J. Lobina Recursion and the competence/performance distinction in AGL tasks , 2011 .

[40]  Stefan Knecht,et al.  Electrical Stimulation of Broca's Area Enhances Implicit Learning of an Artificial Grammar , 2010, Journal of Cognitive Neuroscience.

[41]  Willem J. M. Levelt,et al.  Formal grammars in linguistics and psycholinguistics , 1974 .

[42]  L R Squire,et al.  Intact learning of artificial grammars and intact category learning by patients with Parkinson's disease. , 1999, Behavioral neuroscience.

[43]  A. Friederici Processing local transitions versus long-distance syntactic hierarchies , 2004, Trends in Cognitive Sciences.

[44]  E. Gibson,et al.  Memory Limitations and Structural Forgetting: The Perception of Complex Ungrammatical Sentences as Grammatical , 1999 .

[45]  Arthur S. Reber,et al.  Analogic and abstraction strategies in synthetic grammar learning: A functionalist interpretation , 1978, Cognition.

[46]  Edward E. Smith,et al.  A Parametric Study of Prefrontal Cortex Involvement in Human Working Memory , 1996, NeuroImage.

[47]  Manuel Gimenes,et al.  The effect of noun phrase type on working memory saturation during sentence comprehension , 2009 .

[48]  Joshua B. Tenenbaum,et al.  How recursive is language? A Bayesian exploration , 2010 .

[49]  M. Hauser,et al.  Learning at a distance II. Statistical learning of non-adjacent dependencies in a non-human primate , 2004, Cognitive Psychology.

[50]  L. Squire,et al.  Artificial grammar learning depends on implicit acquisition of both abstract and exemplar-specific information. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[51]  Morten H. Christiansen,et al.  When Less is Less and When Less is More: Starting Small with Staged Input , 2003 .

[52]  J. Elman Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[53]  Morten H. Christiansen,et al.  A Usage-Based Approach to Recursion in Sentence Processing , 2009 .

[54]  E. Newport,et al.  Computation of Conditional Probability Statistics by 8-Month-Old Infants , 1998 .

[55]  Morten H. Christiansen,et al.  Impaired artificial grammar learning in agrammatism , 2010, Cognition.

[56]  M. Hauser,et al.  Segmentation of the speech stream in a non-human primate: statistical learning in cotton-top tamarins , 2001, Cognition.

[57]  Nick Chater,et al.  Phonology impacts segmentation in online speech processing , 2005 .

[58]  Marilyn D. Wang The role of syntactic complexity as a determiner of comprehensibility , 1970 .

[59]  D Burns,et al.  Sentence comprehension and memory for embedded structure , 1977, Memory & cognition.

[60]  Joan Y. Chiao,et al.  An Event-related fMRI Study of Artificial Grammar Learning in a Balanced Chunk Strength Design , 2004, Journal of Cognitive Neuroscience.

[61]  P. Hagoort,et al.  What artificial grammar learning reveals about the neurobiology of syntax , 2012, Brain and Language.

[62]  Dianne Bradley,et al.  The Effect of -ga Sequences on Processing Japanese Multiply Center-Embedded Sentences , 1996, PACLIC.

[63]  Morten H. Christiansen,et al.  On-Line Individual Differences in Statistical Learning Predict Language Processing , 2010, Front. Psychology.

[64]  Willem J.M. Levelt Formal Grammars in Linguistics and Psycholinguistics: Volume 1: An Introduction to the Theory of Formal Languages and Automata, Volume 2: Applications in Linguistic Theory, Volume 3: Psycholinguistic Applications , 2008 .

[65]  Norbert Hornstein,et al.  'The future of language' (Science) [7] , 2005 .

[66]  Z. Pylyshyn The role of competence theories in cognitive psychology , 1973, Journal of psycholinguistic research.

[67]  Morten H. Christiansen,et al.  Sequential learning in non-human primates , 2001, Trends in Cognitive Sciences.

[68]  M. Ullman Contributions of memory circuits to language: the declarative/procedural model , 2004, Cognition.

[69]  Denise Brandão de Oliveira e Britto,et al.  The faculty of language , 2007 .

[70]  William D. Marslen-Wilson,et al.  Crossed and nested dependencies in German and Dutch , 1986 .

[71]  P. Greenfield,et al.  Strategies used to combine seriated cups by chimpanzees (Pan troglodytes), bonobos (Pan paniscus), and capuchins (Cebus apella). , 1999, Journal of comparative psychology.

[72]  Lawrence E. Marks,et al.  Scaling of grammaticalness of self-embedded English sentences , 1968 .

[73]  Jean-Michel Peterfalvi,et al.  L'acceptabilité des phrases , 1971 .

[74]  Peter Hagoort,et al.  A matter of time: Implicit acquisition of recursive sequence structures , 2009 .

[75]  E. Saltzman,et al.  The development of rulebound strategies for manipulating seriated cups: A parallel between action and grammar , 1972 .

[76]  Jeffrey L. Elman,et al.  Distributed Representations, Simple Recurrent Networks, and Grammatical Structure , 1991, Mach. Learn..

[77]  D. Hakes,et al.  Sentence comprehension and relative pronouns , 1970 .

[78]  Joanne M. Williams,et al.  Artificial grammar learning in primary school children with and without developmental dyslexia , 2009, Annals of dyslexia.

[79]  Christopher M. Conway,et al.  Neurocognitive Basis of Implicit Learning of Sequential Structure and Its Relation to Language Processing , 2008, Annals of the New York Academy of Sciences.

[80]  A. Anwander,et al.  The brain differentiates human and non-human grammars: Functional localization and structural connectivity , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[81]  A. Reber Implicit learning of artificial grammars , 1967 .

[82]  Angela D. Friederici,et al.  Hierarchical artificial grammar processing engages Broca's area , 2008, NeuroImage.

[83]  W. Levelt Formal grammars in linguistics and psycholinguistics : Vol.III, Psycholinguistic applications , 1974 .

[84]  R. Gómez,et al.  Infant artificial language learning and language acquisition , 2000, Trends in Cognitive Sciences.

[85]  K. Petersson,et al.  Instruction effects in implicit artificial grammar learning: A preference for grammaticality , 2008, Brain Research.

[86]  P. Hagoort,et al.  The inferior frontal cortex in artificial syntax processing: An rTMS study , 2008, Brain Research.

[87]  G. Miller Some psychological studies of grammar. , 1962 .

[88]  Karl Magnus Petersson,et al.  On cognition, structured sequence processing, and adaptive dynamical systems , 2008 .

[89]  B. Knowlton,et al.  Learning and memory functions of the Basal Ganglia. , 2002, Annual review of neuroscience.

[90]  Morten H. Christiansen The (Non)Necessity of Recursion in Natural Language Processing , 2002 .

[91]  Edith Kaan,et al.  What kinds of syntactic phenomena must biologists, neurobiologists, and computer scientists try to explain and replicate? , 2009 .

[92]  Richard L. Lewis,et al.  Short-term forgetting in sentence comprehension: Crosslinguistic evidence from verb-final structures , 2010 .

[93]  W. Fitch,et al.  Computational Constraints on Syntactic Processing in a Nonhuman Primate , 2004, Science.

[94]  Michael C. Corballis On Phrase Structure and Brain Responses: A Comment on Bahlmann, Gunter, and Friederici (2006) , 2007, Journal of Cognitive Neuroscience.

[95]  Willem H. Zuidema,et al.  Simple rules can explain discrimination of putative recursive syntactic structures by a songbird species , 2009, Proceedings of the National Academy of Sciences.

[96]  Karl Magnus Petersson,et al.  Artificial syntactic violations activate Broca's region , 2004, Cogn. Sci..

[97]  Nick Chater,et al.  Toward a connectionist model of recursion in human linguistic performance , 1999, Cogn. Sci..

[98]  Michael L. Hoover,et al.  Sentence processing strategies in Spanish and English , 1992 .

[99]  S. Kaufman,et al.  Intact implicit learning in autism spectrum conditions , 2010, Quarterly journal of experimental psychology.

[100]  Howard Lasnik,et al.  Syntactic Structures Revisited: Contemporary Lectures on Classic Transformational Theory , 2000 .

[101]  Geoffrey K. Pullum,et al.  Recursion and the infinitude claim , 2010 .

[102]  Thomas F Münte,et al.  Implicit Learning is Intact in Adult Developmental Dyslexic Readers: Evidence from the Serial Reaction Time Task and Artificial Grammar Learning , 2006, Journal of clinical and experimental neuropsychology.