TRACX 2.0: A memory-based, biologically-plausible model of sequence segmentation and chunk extraction

TRACX 2.0: A memory-based, biologically-plausible model of sequence segmentation and chunk extraction Robert M. French (robert.french@u-bourgogne.fr) LEAD-CNRS UMR 5022, Universite de Bourgogne 21000 Dijon, France Garrison W. Cottrell (gary@ucsd.edu) Computer Science and Engineering, UCSD La Jolla, CA 92093-0404, USA Abstract TRACX (French, Addyman, & Mareschal, 2011) is a recursive connectionist system that implicitly extracts chunks from sequence data. It can account for experiments on infant statistical learning and adult implicit learning, as well as real- world phoneme data, and an experiment using backward transitional probabilities that simple recurrent networks cannot account for. One criticism of TRACX, however, is the implausibility in a connectionist model of if-then-else statements. In particular, one of these statements controls what data is copied from the model’s internal memory into its input, based on a hard error threshold. We, therefore, developed a more biologically-plausible version of TRACX devoid of if-then-else statements, relying only on spreading activation and without any learning error threshold. This new model, TRACX 2.0, performs essentially as well as the original TRACX model and, in addition, has two fewer parameters than the original and accounts for the graded nature of chunks. Keywords: chunk extraction; statistical learning; implicit learning; recursive autoassociative memory; autoassociators. Introduction No one disputes that individuals learn to extract structure from their sensory environment. There is, however, a heated debate is over just how this is done. In what follows we will suggest a neurobiologically plausible, memory-based model that achieves this in the auditory domain. The model provides a strong hypothesis as to how people -- infants, as well as adults -- might segment continuous syllable streams into words. The model is an improvement of a recent connectionist memory-based model of sequence segmentation and chunking, TRACX (French, Addyman, & Mareschal, 2011). The new model improves TRACX by removing a crucial if-then-else statement in the model and replaces it with a simple connectionist mechanism. The mainstream view of how segmentation is done, one that has held sway for the nearly two decades, is based on the notion of prediction. This theory supposes that individuals, based on their previous experience with the world, are constantly in the process of making predictions about what is going to happen next in their environment. In so doing, they gradually learn to align their predictions with what actually happens in the world. In order to make these predictions, they must gradually learn the probabilities of successive events in the world. We learn that a flash of lightning will invariably be followed by a clap of thunder, that a “hello” will usually be reciprocated, that a phone call will sometimes be for us, but sometimes not, that the flashing light on a police car will usually be for someone else, but occasionally for us, and so on. This is the basis of the transitional probability (TP) theory of sequence segmentation. The idea is simple. In the syllable stream that an infant hears, many multi-syllable words will be repeated frequently (e.g., bay-bee, mah-mee, bah-tul, and so on) and, as a result, the infant will become better at predicting upcoming within-word syllables compared to upcoming between-word syllables. (The syllable pair bay-bee will be followed by the initial syllable of many different words, whereas as bay will be very frequently followed by bee. The infant thus learns the word bay-bee.) Thus, low syllable-to-syllable TPs (failures to predict) indicate word boundaries. High syllable-to-syllable TPs bind syllables together into words and facilitate their learning. An obvious connectionist candidate for this kind of transitional-probability based learning is the well-known Simple Recurrent Network (SRN, Elman, 1990). While we don’t doubt that prediction is an important aspect of cognition, there are other plausible explanations as to how infants (and adults) learn to segment continuous speech streams into words. Broadly speaking, there are four classes of models used to explain sequence segmentation and word extraction. These are: - Predictive connectionist models , most prominent among them the SRN (Elman, 1990; Cleeremans & McClelland, 1991; Servan-Schreiber, Cleeremans, & McClelland, 1991); - Chunking connectionist models , i.e., TRACX (French, et al., 2011); - Symbolic hybrid models , the best known of which are probably PARSER (Perruchet & Vinter, 1998, 2002) and the Competitive Chunker (Servan-Schreiberr & Anderson, 1990) - Normative statistical models (Frank, Goldwater, Griffiths & Tenenbaum, 2010; Goldwater, Griffiths, & Johnson, 2009; Borschinger, & Johnson, 2011). Recently, Kurumada, Meylan, and Frank (2013) ran a series