Integrating object recognition , visual attention , language and action processing on a robot in a neurobiologically plausible associative architecture

We have implemented a neurobiologically plausible system on a robot that integrates object recognition, visual attention, language and action processing using a coherent cortex-like architecture based on neural associative memories. This system enables the robot to respond to spoken commands like ”bot show plum” or ”bot put apple to yellow cup”. The scenario for this is a robot close to one or two tables carrying certain kinds of fruit and/or other simple objects. Tasks such as finding and pointing to certain fruits in a complex visual scene according to spoken or typed commands can be demonstrated. This involves parsing and understanding of simple sentences, relating the nouns to concrete objects sensed by the camera, and coordinating motor output with planning and sensory processing.

[1]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[2]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[3]  G. Palm.  Neural Assemblies , 1982, Studies of Brain Function.

[4]  Gèunther Palm,et al.  Neural Assemblies: An Alternative Approach to Artificial Intelligence , 1982 .

[5]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[6]  Günther Palm,et al.  Iterative retrieval of sparsely coded associative memory patterns , 1996, Neural Networks.

[7]  Günther Palm,et al.  Improved bidirectional retrieval of sparse patterns stored by Hebbian learning , 1999, Neural Networks.

[8]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[9]  Andreas Knoblauch,et al.  Pattern separation and synchronization in spiking associative memories and visual areas , 2001, Neural Networks.

[10]  Hans Utz,et al.  Miro - middleware for mobile robot applications , 2002, IEEE Trans. Robotics Autom..

[11]  Friedemann Pulvermüller,et al.  The Neuroscience of Language: On Brain Circuits of Words and Serial Order , 2003 .

[12]  Andreas Knoblauch,et al.  Synchronization and pattern separation in spiking associative memories and visual cortical areas , 2004 .

[13]  Friedhelm Schwenker,et al.  Learning Object Recognition in a NeuroBotic System , 2004 .

[14]  G. Palm,et al.  Associating words to visually recognized objects ∗ , 2004 .

[15]  G. Palm,et al.  On associative memory , 2004, Biological Cybernetics.