Artificial grammar learning and neural networks

Artificial Grammar Learning and Neural Networks Karl Magnus Petersson (karl.magnus.petersson@fcdonders.ru.nl) F.C. Donders Centre for Cognitive Neuroimaging, Radboud University Nijmegen, The Netherlands CSI, Center for Intelligent Systems, Universidade do Algarve, Portugal Peter Grenholm (peter_grenholm@yahoo.se) Cognitive Neurophysiology Research Group, Karolinska Institutet 171 76 Stockholm, Sweden Christian Forkstam (christian.forkstam@cns.ki.se) Cognitive Neurophysiology Research Group, Karolinska Institutet 171 76 Stockholm, Sweden transition (Figure 1). The modified RM1 outputs an infinite symbol string: first an end-of-string symbol '#', then a Reber string, a ‘#’, a new string, and so on indefinitely. It turns out that to recognize all possible output strings of RM1, a necessary and sufficient condition is to know a set, TG, of 48 trigrams. A string is generated by RM1 if and only if the string starts with ‘#’ and only contains trigrams in TG. To show this, we observe that the Reber grammar yields exactly the same strings as RM2 (Figure 2). Assume that we know RM1 and that we know the latest two symbols in an output string from RM2. This determines the internal state of RM1 and RM2. The set of possible bigrams is {MT,MV,…,#V}. Abstract Recent FMRI studies indicate that language related brain regions are engaged in artificial grammar (AG) processing. In the present study we investigate the Reber grammar by means of formal analysis and network simulations. We outline a new method for describing the network dynamics and propose an approach to grammar extraction based on the state-space dynamics of the network. We conclude that statistical frequency-based and rule-based acquisition procedures can be viewed as complementary perspectives on grammar learning, and more generally, that classical cognitive models can be viewed as a special case of a dynamical systems perspective on information processing. Keywords: artificial grammar learning, neural network, dynamical systems. Introduction T According to Chomsky a core feature of natural language processing is the ‘infinite use of finite means’. The family of right-linear phrase structure grammars, implementable in the finite-state architecture (FSA), is a simple formal model of this idea. The work of Reber (1967) suggested that humans can learn AGs implicitly and that the relevant structure is abstracted from the input. Reber (1967) proposed that this process is intrinsic to natural language learning and it has been suggested that AG learning (AGL) is a relevant model for aspects of language acquisition (Gomez & Gerken, 2000). Recent FMRI studies indicate that language related brain regions are engaged in AG processing (Petersson et al., 2004). Here we investigate the Reber grammar (Figure 1) by means of formal analysis and network simulations and outline an approach to grammar extraction based on the network state-space dynamics. Elementary formal analysis We begin by showing that the Reber grammar, and in certain respects similar AGs, can be learned by acquiring a finite set of n-grams (for details see Grenholm, 2003). To this end, we modify the Reber machine (RM0) by connecting the final to the initial state with a #-labeled V M T X R M V X R Figure 1: The transition graph representation of RM1 corresponding to the Reber grammar. We obtain the set of possible trigrams from RM2 if we take these 21 bigrams and extend them according to Figure 2, yielding TG={MTT,MTV,…,#VX}. It is clear that a string is an output of RM1, if it starts with ‘#’ and only contains trigrams in TG. Conversely, assume that a string begins with one of the 48 trigrams, the first symbol of which is ‘#’. The only possibilities are ‘#MT’, ‘#MV’, and ‘#VX’, which