Symbol Grounding With Delay Coordinates

"There is nothing more basic than categorization to our thought, perception, action, and speech" (Lakoff 1984). Moreover, categories of sensory experience provide the semantic glue between the world and the meaningless symbols often used to represent those experiences. As such, the focus of this work is an unsupervised learning mechanism for extracting categories from time series. We have in mind the situation where a sensorimotor agent, such as an infant or mobile robot, records streams of sensor readings while interacting with a complex environment. To make the leap from percepts to symbolic thought and language, the agent requires a way of transforming uninterpreted sensor information into meaningful categories. That is, the agent must solve the bottom-up version of the symbol grounding problem (Harnad 1990). The solution outlined below was inspired by the method of delays, a nonlinear dynamics tool for producing spatial representations of time-based data. One general technique for discovering categories is to form clusters of points in a suitable space. This was the basis of Elman’s work on learning lexical classes from word co-occurrence statistics (Elman 1990). Elman first trained a recurrent neural network to predict successive words in a long input string. This then set the stage for hierarchical clustering of the hidden-unit activation space, where the result was groups of words that coincide with classes like NOUN-FOOD or VERB-PERCEPT. But how can we ground such syntactic classes in sensorimotor interaction with the environment? We answer this question, not with recurrent neural networks, which require a great deal of training, but with the method of delays, which maps a snippet of time series data to a point in delay-coordinate space. Delay coordinates are just successive sensor readings taken at a suitable time interval (Takeus 1981; Roseustein, Collins, & De Luca 1994). For instance, imagine that our mobile robot is two meters from an obstacle and its sonar reports the value 2015 ram. From this single reading there is no way to know if the robot will crash into the obstacle. But suppose the sonar reports 9-140 mm a second later. We could plot the point