Capturing Gradience, Continuous Change, and Quasi‐Regularity in Sound, Word, Phrase, and Meaning

One vision of the nature of language holds that a language consists of a set of symbolic unit types, and a set of units of each type, together with a set of grammatical principles that constrain how these units can be used to compose other units, and a system of rules that project structured arrangements of such units onto other structured arrangements of units (for example, from syntactic to semantic structure). An alternative vision of the nature of language holds that it is often useful to characterize language as if the above statements were true, but only as a way of approximately notating or summarizing aspects of language. In reality, according to this alternative vision, approximate conformity to structured systems of symbolic units and rules arises historically, developmentally, and in the moment, from the processes that operate as users communicate with each other using sound or gesture as their medium of communication. These acts of communication leave residues that can be thought of as storing knowledge in the form of the continuous-valued parameters of a complex dynamical system (i.e. a system characterized by continuous, stochastic, and non-linear differential equations). Greatly influenced by the work of Joan Bybee (1985, 2001) and others who have pointed out some of its advantages, I am a disciple of this alternative vision (Bybee and McClelland, 2005; McClelland and Bybee, 2007). As argued in the Bybee and McClelland papers just cited, neural network models that rely on distributed representations (sometimes called connectionist or parallel-distributed processing models) provide one useful way of capturing features of this vision. Such models are, in general, just the sort of continuous, stochastic, non-linear systems that are needed to capture the key phenomena, and the connection weights and other variables in such networks are the continuous-valued parameters in which