Integrating Distributional, Prosodic and Phonological Information in a Connectionist Model of Language Acquisition

Integrating Distributional, Prosodic and Phonological Information in a Connectionist Model of Language Acquisition Morten H. Christiansen†‡ (morten@siu.edu) Rick A.C. Dale† (racdale@siu.edu) †Department of Psychology ‡Department of Linguistics Carbondale, IL 62901 USA Abstract Children acquire the syntactic structure of their native lan- guage with remarkable speed and reliability. Recent work in developmental psycholinguistics suggests that children may bootstrap grammatical categories and basic syntactic structure by exploiting distributional, phonological, and prosodic cues. However, these cues are probabilistic, and are individually unreliable. In this paper, we present a series of simulations exploring the integration of mul- tiple probabilistic cues in a connectionist model. The first simulation demonstrates that multiple-cue integra- tion promotes significantly better, faster, and more uni- form acquisition of syntax. In a second simulation, we show how this model can also accommodate recent data concerning the sensitivity of young children to prosody and grammatical function words. Our third simulation il- luminates the potential contribution of prenatal language experience to the acquisition of syntax through multiple- cue integration. Finally, we demonstrate the robustness of the multiple-cue model in the face of potentially distract- ing cues, uncorrelated with grammatical structure. Introduction Before children can ride a bicycle or tie their shoes, they have learned a great deal about how words are combined to form complex sentences. This achievement is espe- cially impressive because children acquire most of this syntactic knowledge with little or no direct instruction. Nevertheless, mastering natural language syntax may be among the most difficult learning tasks that children face. In adulthood, syntactic knowledge can be characterized by constraints governing the relationship between gram- matical categories of words (such as noun and verb) in a sentence. But acquiring this knowledge presents the child with a “chicken-and-egg” problem: the syn- tactic constraints presuppose the grammatical categories in terms of which they are defined; and the validity of grammatical categories depends on how far they support syntactic constraints. A similar “bootstrapping” prob- lem faces a student learning an academic subject such as physics: understanding momentum or force presup- poses some understanding of the physical laws in which they figure, yet these laws presuppose these very con- cepts. But the bootstrapping problem solved by young children seems vastly more challenging, both because the constraints governing natural language are so intri- cate, and because young children do not have the in- tellectual capacity or explicit instruction available to the academic student. Determining how children accomplish the astonishing feat of language acquisition remains a key question in cognitive science. By 12 months, infants are attuned to the phonolog- ical and prosodic regularities of their native language (Jusczyk, 1997; Kuhl, 1999). This perceptual attunement may provide an essential scaffolding for later learning by biasing children toward aspects of the input that are par- ticularly informative for acquiring grammatical informa- tion. Specifically, we hypothesize that integrating multi- ple probabilistic cues (phonological, prosodic and distri- butional) by perceptually attuned general-purpose learn- ing mechanisms may hold the key to how children solve the bootstrapping problem. Multiple cues can provide re- liable evidence about linguistic structure that is unavail- able from any single source of information. In the remainder of this paper, we first review empir- ical evidence suggesting that infants may use a combi- nation of distributional, phonological and prosodic cues to bootstrap into language. We then report a series of simulations, demonstrating the efficacy of multiple-cue integration within a connectionist framework. Simula- tion 1 shows how multiple-cue integration results in bet- ter, faster and more uniform learning. Simulation 2 es- tablishes that the trained three-cue networks are able to mimic the effect of grammatical and prosodic manipula- tions in a sentence comprehension study with 2-year-olds (Shady & Gerken, 1999). Simulation 3 reveals how pre- natal exposure to gross-level phonological and prosodic input facilitates postnatal learning within the multiple- cue integration framework. Finally, Simulation 4 demon- strates that adding additional distracting cues, irrelevant to the syntactic acquisition task, does not hinder learning. Cues Available for Syntax Acquisition Although some kind of innate knowledge may play a role in language acquisition, it cannot solve the boot- strapping problem. Even with built-in abstract knowl- edge about grammatical categories and syntactic rules (e.g., Pinker, 1984), the bootstrapping problem remains formidable: children must map the right sound strings onto the right grammatical categories while determining the specific syntactic relations between these categories in their native language. Moreover, the item-specific na- ture of early syntactic productions challenges the use- fulness of hypothesized innate grammatical categories