Innite Languages, Finite Minds Connectionism, Learning and Linguistic Structure

This thesis presents a connectionist theory of how infinite languages may fit within finite minds. Arguments are presented against the distinction between linguistic competence and observable language performance. It is suggested that certain kinds of finite state automata—i.e., recurrent neural networks—are likely to have sufficient computational power, and the necessary generalization capability, to serve as models for the processing and acquisition of linguistic structure. These arguments are further corroborated by a number of computer simulations, demonstrating that recurrent connectionist models are able to learn complex recursive regularities and have powerful generalization abilities. Importantly, the performance evinced by the networks are comparable with observed human behavior on similar aspects of language. Moreover, an evolutionary account is provided, advocating a learning and processing based explanation of the origin and subsequent phylogenetic development of language. This view construes language as a nonobligate symbiant, arguing that language has evolved to fit human learning and processing mechanisms, rather than vice versa. As such, this perspective promises to explain linguistic universals in functional terms, and motivates an account of language acquisition which incorporates innate, but not language-specific constraints on the learning process. The purported poverty of the stimulus is re-appraised in this light, and it is concluded that linguistic structure may be learnable by bottom-up statistical learning models, such as, connectionist neural networks.

[1]  J. Baldwin A New Factor in Evolution , 1896, The American Naturalist.

[2]  ダーウィン チャールス,et al.  The descent of man and selection in relation to sex , 1907 .

[3]  Noam Chomsky,et al.  On Certain Formal Properties of Grammars , 1959, Inf. Control..

[4]  Willard Van Orman Quine,et al.  Word and Object , 1960 .

[5]  G. Miller Some psychological studies of grammar. , 1962 .

[6]  G. A. Miller,et al.  Finitary models of language users , 1963 .

[7]  George A. Miller,et al.  Free Recall of Self-Embedded English Sentences , 1964, Inf. Control..

[8]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[9]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[10]  Walter S. Stolz,et al.  A study of the ability to decode grammatically novel sentences , 1967 .

[11]  Lawrence E. Marks,et al.  Scaling of grammaticalness of self-embedded English sentences , 1968 .

[12]  P. A. Reich The Finiteness of Natural Language , 1969 .

[13]  T. Kuhn The Structure of Scientific Revolutions 2nd edition , 1970 .

[14]  Donald J. Foss,et al.  Some effects of memory limitation upon sentence comprehension and recall , 1970 .

[15]  J. Kimball Seven principles of surface structure parsing in natural language , 1973 .

[16]  Philip Lieberman,et al.  On the evolution of language: A unified view , 1973 .

[17]  M. Braine,et al.  Short-term memory limitations on decoding self-embedded sentences , 1974 .

[18]  Paul Kiparsky,et al.  HISTORICAL LINGUISTICS AND THE ORIGIN OF LANGUAGE , 1976 .

[19]  P. Liebermann Interactive models for evolution: neural mechanisms, anatomy, and behavior. , 1976, Annals of the New York Academy of Sciences.

[20]  Michael Don Palmer,et al.  Reflections on language , 1977 .

[21]  D Burns,et al.  Sentence comprehension and memory for embedded structure , 1977, Memory & cognition.

[22]  J. M. Smith,et al.  Optimization Theory in Evolution , 1978 .

[23]  M. Marcus A Computational Account of Some Constraints on Language , 1978, CL.

[24]  R. Haber,et al.  Visual Perception , 2018, Encyclopedia of Database Systems.

[25]  Janet D. Fodor,et al.  The sausage machine: A new two-stage parsing model , 1978, Cognition.

[26]  Mitchell P. Marcus,et al.  A theory of syntactic recognition for natural language , 1979 .

[27]  Mitchell P. Marcus,et al.  A Computational Account of Some Constraints on Language , 1986, TINLAP.

[28]  S. Gould Panselectionist pitfalls in Parker & Gibson's model for the evolution of intelligence , 1979 .

[29]  S. Gould,et al.  The spandrels of San Marco and the Panglossian paradigm: a critique of the adaptationist programme , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[30]  Noam Chomsky Language and Responsibility , 1979 .

[31]  E. Wanner The ATN and the sausage machine: Which one is baloney? , 1980, Cognition.

[32]  Morris Halle,et al.  The rules of language , 1980, IEEE Transactions on Professional Communication.

[33]  Murray Grossman,et al.  A central processor for hierarchically-structured material: Evidence from broca's aphasia , 1980, Neuropsychologia.

[34]  F Grosjean,et al.  Spoken word recognition processes and the gating paradigm , 1980, Perception & psychophysics.

[35]  Noam Chomsky,et al.  Lectures on Government and Binding , 1981 .

[36]  Bryan J. Hubbell,et al.  On memory limitations in natural language processing , 1982 .

[37]  Geoffrey Sampson,et al.  A myth about centre-embedding , 1982 .

[38]  S. Gould,et al.  Exaptation—a Missing Term in the Science of Form , 1982, Paleobiology.

[39]  Robert William Milne,et al.  Predicting Garden Path Sentences , 1982, Cogn. Sci..

[40]  Geoffrey K. Pullum,et al.  Natural languages and context-free languages , 1982 .

[41]  Lyn Frazier,et al.  The interaction of syntax and semantics during sentence processing: eye movements in the analysis of semantically biased sentences , 1983 .

[42]  C. L. Baker,et al.  The Logical problem of language acquisition , 1984 .

[43]  J. Michael Verbal behavior. , 1984, Journal of the experimental analysis of behavior.

[44]  Steven Pinker,et al.  Language learnability and language development , 1985 .

[45]  Mark Steedman,et al.  On not being led up the garden path : The use of context by the psychological syntax processor , 1985 .

[46]  C Snow,et al.  Child language data exchange system , 1984, Journal of Child Language.

[47]  Geoffrey K. Pullum,et al.  Generalized Phrase Structure Grammar , 1985 .

[48]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[49]  C. Clifton,et al.  The independence of syntactic processing , 1986 .

[50]  Stephen Pulman,et al.  Grammars, parsers, and memory limitations , 1986 .

[51]  William D. Marslen-Wilson,et al.  Crossed and nested dependencies in German and Dutch , 1986 .

[52]  M. Land Visual Perception: Physiology, Psychology and Ecology, Vicki Bruce, Patrick Green. Lawrence Erlbaum, London (1985), xiii, +369. Price £8.95 (paperback) , 1986 .

[53]  James L. McClelland,et al.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition : Psychological and Biological Models , 1986 .

[54]  Robert C. Berwick,et al.  The Grammatical Basis of Linguistic Performance: Language Use and Acquisition , 1986 .

[55]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[56]  Mark Steedman,et al.  Combinatory grammars and parasitic gaps , 1987 .

[57]  Noam Chomsky Language and Problems of Knowledge , 1987 .

[58]  John Maynard Smith,et al.  When learning guides evolution , 1987, Nature.

[59]  Steven Pinker,et al.  Productivity and constraints in the acquisition of the passive , 1987, Cognition.

[60]  Patricia K. Kuhl,et al.  The special-mechanisms debate in speech research: Categorization tests on animals and infants. , 1987 .

[61]  W. Schneider Connectionism: Is it a paradigm shift for psychology? , 1987 .

[62]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[63]  Mark Steedman,et al.  Interaction with context during human sentence processing , 1988, Cognition.

[64]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[65]  Steven Pinker,et al.  On language and connectionism , 1988 .

[66]  Steven Pinker,et al.  Rules and connections in human language , 1988, Trends in Neurosciences.

[67]  E. Capaldi,et al.  The rat’s simultaneous anticipation of remote events and current events can be sustained by event memories alone , 1988 .

[68]  P. Smolensky THE CONSTITUENT STRUCTURE OF CONNECTIONIST MENTAL STATES: A REPLY TO FODOR AND PYLYSHYN , 2010 .

[69]  P. Smolensky On the proper treatment of connectionism , 1988, Behavioral and Brain Sciences.

[70]  Noam Chomsky,et al.  Language and problems of knowledge : the Managua lectures , 1990 .

[71]  A. Reber,et al.  Implicit learning: Robustness in the face of psychiatric disorders , 1988, Journal of psycholinguistic research.

[72]  S. Pinker Learnability and Cognition: The Acquisition of Argument Structure , 1989 .

[73]  A M Liberman,et al.  A specialization for speech perception. , 1989, Science.

[74]  M. Piattelli-Palmarini Evolution, selection and cognition: From “learning” to parameter setting in biology and in the study of language , 1989, Cognition.

[75]  Daniel C. Dennett,et al.  Mother Nature versus the Walking Encyclopedia , 1989 .

[76]  James L. McClelland,et al.  Finite State Automata and Simple Recurrent Networks , 1989, Neural Computation.

[77]  V. Bruce,et al.  Visual perception: Physiology, psychology and ecology, 2nd ed. , 1990 .

[78]  David Kirsh,et al.  When is Information Explicitly Represented , 1990 .

[79]  Tim van Gelder,et al.  Compositionality: A Connectionist Variation on a Classical Theme , 1990, Cogn. Sci..

[80]  Eric Brill,et al.  Deducing Linguistic Structure from the Statistics of Large Corpora , 1990, HLT.

[81]  M. Gopnik,et al.  Genetic basis of grammar defect , 1990, Nature.

[82]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[83]  J. Fodor,et al.  Connectionism and the problem of systematicity: Why Smolensky's solution doesn't work , 1990, Cognition.

[84]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[85]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[86]  Tim van Gelder Connectionism and Language Processing , 1990, ÖGAI.

[87]  N. Chater,et al.  Autonomy, implementation and cognitive architecture: A reply to Fodor and Pylyshyn , 1990, Cognition.

[88]  P. Fletcher,et al.  Speech and language defects , 1990, Nature.

[89]  Peter M. Duppenthaler Maturational Constraints on Language Learning , 1990 .

[90]  David J. Chalmers,et al.  Syntactic Transformations on Distributed Representations , 1990 .

[91]  L. Cosmides,et al.  Toward an adaptationist psycholinguistics , 1990, Behavioral and Brain Sciences.

[92]  M. Gopnik Feature-blind grammar and dysphasia , 1990, Nature.

[93]  S. Pinker,et al.  Natural language and natural selection , 1990, Behavioral and Brain Sciences.

[94]  D. Chalmers Why Fodor and Pylyshyn Were Wrong: The Simplest Refutation , 1990 .

[95]  Arthur S. Reber,et al.  On the primacy of the implicit: Comment on Perruchet and Pacteau. , 1990 .

[96]  L. Gleitman,et al.  [Language and thought]. , 1991, La Revue du praticien.

[97]  M. Gopnik,et al.  Familial aggregation of a developmental language disorder , 1991, Cognition.

[98]  P. Lieberman Speech and brain evolution , 1991, Behavioral and Brain Sciences.

[99]  M. Just,et al.  Individual differences in syntactic processing: The role of working memory , 1991 .

[100]  P. Kuhl Human adults and human infants show a “perceptual magnet effect” for the prototypes of speech categories, monkeys do not , 1991, Perception & psychophysics.

[101]  S. Crain Language acquisition in the absence of experience , 1991, Behavioral and Brain Sciences.

[102]  Debatable constraints , 1991, Behavioral and Brain Sciences.

[103]  Dan I. Slobin,et al.  Can Crain constrain the constraints? , 1991, Behavioral and Brain Sciences.

[104]  T. Gelder,et al.  Classical Questions, Radical Answers: Connectionism and the Structure of Mental Representations , 1991 .

[105]  Catherine L. Harris,et al.  Alternatives to linguistic arbitrariness , 1991, Behavioral and Brain Sciences.

[106]  Richard Hudson,et al.  English word grammar , 1995 .

[107]  Ianthi Maria Tsimpli,et al.  Linguistic modularity? A case study of a 'Savant' linguist* , 1991 .

[108]  James R. Hurford,et al.  The evolution of the critical period for language acquisition , 1991, Cognition.

[109]  M. Hauser If you've got it, why not flaunt it? Monkeys with Broca's area but no syntactical structure to their vocal utterances , 1991, Behavioral and Brain Sciences.

[110]  Tetsuro Matsuzawa,et al.  Nesting cups and metatools in chimpanzees , 1991, Behavioral and Brain Sciences.

[111]  B. Jacobs Neurobiology and language acquisition: Continuity and identity , 1991 .

[112]  S. Laughlin,et al.  Computational neuroethology: a provisional manifesto , 1991 .

[113]  A. Stolcke Syntactic Category Formation with Vector Space Grammars , 1991 .

[114]  Jeffrey L. Elman,et al.  Distributed Representations, Simple Recurrent Networks, and Grammatical Structure , 1991, Mach. Learn..

[115]  Robert F. Port,et al.  Representing Aspects of Language , 1991 .

[116]  Dennis Norris,et al.  A dynamic-net model of human speech recognition , 1991 .

[117]  P. Greenfield,et al.  Language, tools and brain: The ontogeny and phylogeny of hierarchically organized sequential behavior , 1991, Behavioral and Brain Sciences.

[118]  Jeffrey L. Elman,et al.  A PDP Approach to Processing Center-Embedded Sentences , 1992 .

[119]  E. Bates,et al.  Symbols and Syntax: A Darwinian Approach to Language Development , 1992 .

[120]  A. Reber An evolutionary context for the cognitive unconscious , 1992 .

[121]  Annette Karmiloff-Smith,et al.  Nature, Nurture and PDP: Preposterous Developmental Postulates? , 1992 .

[122]  M. Corballis On the evolution of language and generativity , 1992, Cognition.

[123]  A. Narayanan Is Connectionism Compatible with Rationalism , 1992 .

[124]  Frederick Jelinek,et al.  Basic Methods of Probabilistic Context Free Grammars , 1992 .

[125]  Nick Chater,et al.  Connectionism, Learning and Meaning , 1992 .

[126]  Lars Niklasson,et al.  Connectionism and the Issues of Compositionality and Systematicity , 1992 .

[127]  Noel E. Sharkey,et al.  Functional Compositionality and Soft Preference Rules , 1992 .

[128]  David Kirsh,et al.  PDP Learnability and Innate Knowledge of Language , 1992 .

[129]  Nick Chater,et al.  FINDING LINGUISTIC STRUCTURE WITH RECURRENT NEURAL NETWORKS , 1992 .

[130]  K. Stevens,et al.  Linguistic experience alters phonetic perception in infants by 6 months of age. , 1992, Science.

[131]  Barbara Hannan,et al.  Connectionism and the Mind: An Introduction to Parallel Processing in Networks , 1992 .

[132]  Zenon W. Pylyshyn,et al.  Connectionism and cognitive architecture , 1993 .

[133]  A. Reber Implicit learning and tacit knowledge , 1993 .

[134]  Stephen Jay Gould,et al.  Eight Little Piggies: Reflections in Natural History , 1993 .

[135]  Andrew S. Noetzel,et al.  Sequence Recognition with Recurrent Neural Networks , 1993 .

[136]  V. Ramachandran,et al.  Behavioral and magnetoencephalographic correlates of plasticity in the adult human brain. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[137]  Axel Cleeremans,et al.  Mechanisms of Implicit Learning: Connectionist Models of Sequence Processing , 1993 .

[138]  J. Elman Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[139]  Nick Chater,et al.  DISTRIBUTIONAL INFORMATION AND THE ACQUISITION OF LINGUISTIC CATEGORIES - A STATISTICAL APPROACH , 1993 .

[140]  S. Quartz Neural networks, nativism, and the plausibility of constructivism , 1993, Cognition.

[141]  V. Marchman,et al.  From rote learning to system building: acquiring verb morphology in children and connectionist nets , 1993, Cognition.

[142]  Bruce J. MacLennan,et al.  Characteristics of connectionist knowledge representation , 1991, Inf. Sci..

[143]  Robert F. Hadley Systematicity in Connectionist Language Learning , 1994 .

[144]  R. French,et al.  Genes, Phenes and the Baldwin Effect: Learning and Evolution in a Simulated Population , 1994 .

[145]  Massimo Piattelli-Palmarini,et al.  Ever since language and learning: afterthoughts on the Piaget-Chomsky debate , 1994, Cognition.

[146]  G. Kane Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[147]  Ashwin Ram,et al.  Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society , 2019 .

[148]  Morten H. Christiansen,et al.  Generalization and connectionist language learning , 1994 .

[149]  Mark S. Seidenberg Language and connectionism: the developing interface , 1994, Cognition.

[150]  Michael C. Corballis,et al.  The generation of generativity: a response to Bloom , 1994, Cognition.

[151]  P. Bloom,et al.  Generativity within language and other cognitive domains , 1994, Cognition.

[152]  Kow Kuroda Is the Ontological Commitment to "Rules" Really Preferable to Linguistic Science? : A Critical Assessment of the Symbolist-Connectionist Debate on the "Learning of the Past Tenses of English Verbs , 1995 .

[153]  J. Elman,et al.  Learning and morphological change , 1995, Cognition.

[154]  S. Pinker The language instinct : how the mind creates language , 1995 .

[155]  Elizabeth Bates,et al.  Inducing Agrammatic Profiles in Normals: Evidence for the Selective Vulnerability of Morphology under Cognitive Resource Limitation , 1995, Journal of Cognitive Neuroscience.

[156]  Martin Davies,et al.  Two Notions of Implicit Rules , 1995 .

[157]  Steven Phillips,et al.  Connectionism and the problem of systematicity , 1995 .

[158]  Matthew W. Crocker,et al.  Mechanisms for Sentence Processing , 1996 .

[159]  M Bates,et al.  Generative Grammar , 2022 .