A dynamic network analysis of emergent grammar

For languages to survive as complex cultural systems, they need to be learnable. According to traditional approaches, learning is made possible by constraining the degrees of freedom in advance of experience and by the construction of complex structure during development. This article explores a third contributor to complexity: namely, the extent to which syntactic structure can be an emergent property of how simpler entities – words – interact with one another. The authors found that when naturalistic child directed speech was instantiated in a dynamic network, communities formed around words that were more densely connected with other words than they were with the rest of the network. This process is designed to mirror what we know about distributional patterns in natural language: namely, the network communities represented the syntactic hubs of semi-formulaic slot-and-frame patterns, characteristic of early speech. The network itself was blind to grammatical information and its organization reflected (a) the frequency of using a word and (b) the probabilities of transitioning from one word to another. The authors show that grammatical patterns in the input disassociate by community structure in the emergent network. These communities provide coherent hubs which could be a reliable source of syntactic information for the learner. These initial findings are presented here as proof-of-concept in the hope that other researchers will explore the possibilities and limitations of this approach on a larger scale and with more languages. The implications of a dynamic network approach are discussed for the learnability burden and the development of an adult-like grammar.

[1]  Mark E. J. Newman,et al.  Structure and Dynamics of Networks , 2009 .

[2]  Alexandre Arenas,et al.  Semantic Networks: Structure and Dynamics , 2010, Entropy.

[3]  James L. McClelland,et al.  Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences , 1986 .

[4]  Joan L. Bybee,et al.  Frequency of Use and the Organization of Language , 2006 .

[5]  Ulrich Eggers,et al.  Emergence From Chaos To Order , 2016 .

[6]  Toben H. Mintz Frequent frames as a cue for grammatical categories in child directed speech , 2003, Cognition.

[7]  J. Pine,et al.  Lexically-based learning and early grammatical development , 1997, Journal of Child Language.

[8]  Dan Roth,et al.  Baby SRL: Modeling Early Language Acquisition , 2008, CoNLL.

[9]  Thomas T. Hills,et al.  Small Worlds and Semantic Network Growth in Typical and Late Talkers , 2011, PloS one.

[10]  Joao Antonio Pereira,et al.  Linked: The new science of networks , 2002 .

[11]  Olaf Sporns,et al.  Networks analysis, complexity, and brain function , 2002 .

[12]  Noam Chomsky,et al.  The Logical Structure of Linguistic Theory , 1975 .

[13]  Dan Klein,et al.  Natural language grammar induction with a generative constituent-context model , 2005, Pattern Recognit..

[14]  Mark Steyvers,et al.  The Large-Scale Structure of Semantic Networks , 2005 .

[15]  O. Sporns Network Analysis , Complexity , and Brain Function , 2002 .

[16]  P. Kuhl A new view of language acquisition. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[18]  Ellen M. Markman,et al.  Constraints on word meaning in early language acquisition , 1994 .

[19]  Khalil Sima'an,et al.  Data-Oriented Parsing , 2003 .

[20]  P. Kuhl Early language acquisition: cracking the speech code , 2004, Nature Reviews Neuroscience.

[21]  Rens Bod,et al.  Is the End of Supervised Parsing in Sight? , 2007, ACL.

[22]  Joshua B. Tenenbaum,et al.  The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth , 2001, Cogn. Sci..

[23]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[24]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[25]  R. Langacker Foundations of cognitive grammar , 1983 .

[26]  Morten H. Christiansen,et al.  Language Learning as Language Use: A Cross-Linguistic Model of Child Language Development , 2019, Psychological review.

[27]  J. Elman,et al.  Language input and semantic categories: a relation between cognition and early word learning , 2006, Journal of Child Language.

[28]  J. Elman,et al.  Rethinking Innateness: A Connectionist Perspective on Development , 1996 .

[29]  T. A. Cartwright,et al.  Syntactic categorization in early language acquisition: formalizing the role of distributional analysis , 1997, Cognition.

[30]  M. Ross Quillian,et al.  Retrieval time from semantic memory , 1969 .

[31]  Toben H. Mintz Category induction from distributional cues in an artificial language , 2002, Memory & cognition.

[32]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[33]  C. Mervis,et al.  Early object labels: the case for a developmental lexical principles framework , 1994, Journal of Child Language.

[34]  Amos Maritan,et al.  Size and form in efficient transportation networks , 1999, Nature.

[35]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[36]  T. Givon Functionalism and Grammar , 1995 .

[37]  B. MacWhinney The Acquisition Of Morphophonology , 1978 .

[38]  Michael Tomasello,et al.  A construction based analysis of child directed speech , 2003, Cogn. Sci..

[39]  Zellig S. Harris,et al.  Distributional Structure , 1954 .

[40]  Kenneth Wexler,et al.  Formal Principles of Language Acquisition , 1980 .

[41]  R. Mccall,et al.  The Genetic and Environmental Origins of Learning Abilities and Disabilities in the Early School , 2007, Monographs of the Society for Research in Child Development.

[42]  Thomas T. Hills,et al.  Categorical structure among shared features in networks of early-learned nouns , 2009, Cognition.

[43]  Leonard Bloomfield,et al.  Language or ideas , 1936 .

[44]  Fernand Gobet,et al.  Simulating the cross-linguistic pattern of Optional Infinitive errors in children’s declaratives and Wh- questions , 2015, Cognition.

[45]  Ruth Clark Performing without competence , 1974, Journal of Child Language.

[46]  Jeffrey Lidz,et al.  When Domain-General Learning Fails and When It Succeeds: Identifying the Contribution of Domain Specificity , 2009 .

[47]  Nick Chater,et al.  Distributional Information: A Powerful Cue for Acquiring Syntactic Categories , 1998, Cogn. Sci..

[48]  Ramon Ferrer i Cancho,et al.  The small world of human language , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[49]  Suzanne Stevenson,et al.  A Computational Model of Early Argument Structure Acquisition , 2008, Cogn. Sci..

[50]  Julian M. Pine,et al.  Constructing a Language: A Usage-Based Theory of Language Acquisition. , 2004 .

[51]  Elissa L. Newport,et al.  The distributional structure of grammatical categories in speech to young children , 2002, Cogn. Sci..

[52]  A. McKane,et al.  Goldilocks Forgetting in Cross-Situational Learning , 2018, Front. Psychol..

[53]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[54]  Michael Tomasello,et al.  Two-year-old children's production of multiword utterances: A usage-based analysis , 2009 .

[55]  J. Elman Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[56]  Elissa L. Newport,et al.  Statistical Learning of Syntax: The Role of Transitional Probability , 2007 .

[57]  James L. McClelland,et al.  Explorations in parallel distributed processing: a handbook of models, programs, and exercises , 1988 .

[58]  M. Tomasello,et al.  Modeling children's early grammatical knowledge , 2009, Proceedings of the National Academy of Sciences.

[59]  Dan Roth,et al.  Minimally Supervised Model of Early Language Acquisition , 2009, CoNLL.

[60]  Thomas L. Griffiths,et al.  A fully Bayesian approach to unsupervised part-of-speech tagging , 2007, ACL.

[61]  W. Singer,et al.  Selection of intrinsic horizontal connections in the visual cortex by correlated neuronal activity. , 1992, Science.

[62]  William Croft,et al.  Radical Construction Grammar: Syntactic Theory in Typological Perspective , 2001 .

[63]  Richard N. Aslin,et al.  From shared contexts to syntactic categories: The role of distributional information in learning linguistic form-classes , 2013, Cognitive Psychology.

[64]  Hinrich Schütze,et al.  Part-of-Speech Induction From Scratch , 1993, ACL.

[65]  A. Goldberg Constructions at Work: The Nature of Generalization in Language , 2006 .

[66]  W. Bruce Croft,et al.  Language Is a Complex Adaptive System: Position Paper , 2009 .

[67]  Nick Chater,et al.  Distributional Bootstrapping: From Word Class to Proto-Sentence , 2019, Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society.

[68]  S. Pinker Formal models of language learning , 1979, Cognition.

[69]  M. Tomasello,et al.  Early syntactic creativity: a usage-based approach. , 2003, Journal of child language.

[70]  J. Elman Connectionist models of cognitive development: where next? , 2005, Trends in Cognitive Sciences.

[71]  Morten H. Christiansen,et al.  Connectionist psycholinguistics: capturing the empirical data , 2001, Trends in Cognitive Sciences.

[72]  E. Newport,et al.  WORD SEGMENTATION : THE ROLE OF DISTRIBUTIONAL CUES , 1996 .

[73]  Afsaneh Fazly,et al.  An Incremental Bayesian Model for Learning Syntactic Categories , 2008, CoNLL.

[74]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[75]  Anna L. Theakston,et al.  The role of performance limitations in the acquisition of verb-argument structure: an alternative account. , 2001, Journal of child language.