Sentence Understanding and Learning of New Words with Large-Scale Neural Networks

We have implemented a speech command system which can understand simple command sentences like "Bot lift ball" or "Bot go table" using hidden Markov models (HMMs) and associative memories with sparse distributed representations. The system is composed of three modules: (1) A set of HMMs is used on phoneme level to get a phonetic transcription of the spoken sentence, (2) a network of associative memories is used to determine the word belonging to the phonetic transcription and (3) a neural network is used on the sentence level to determine the meaning of the sentence. The system is also able to learn new object words during performance.

[1]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[2]  Günther Palm,et al.  Modelling of syntactical processing in the cortex , 2007, Biosyst..

[3]  Andreas Knoblauch,et al.  Pattern separation and synchronization in spiking associative memories and visual areas , 2001, Neural Networks.

[4]  Günther Palm,et al.  Iterative retrieval of sparsely coded associative memory patterns , 1996, Neural Networks.

[5]  Jonathan G. Fiscus,et al.  Darpa Timit Acoustic-Phonetic Continuous Speech Corpus CD-ROM {TIMIT} | NIST , 1993 .

[6]  Deb K. Roy,et al.  Learning visually grounded words and syntax for a scene description task , 2002, Comput. Speech Lang..

[7]  Günther Palm,et al.  Biomimetic Neural Learning for Intelligent Robots - Intelligent Systems, Cognitive Robotics, and Neuroscience , 2005, Biomimetic Neural Learning for Intelligent Robots.

[8]  Aude Billard,et al.  DRAMA, a Connectionist Architecture for Control and Learning in Autonomous Robots , 1999, Adapt. Behav..

[9]  Michael A. Arbib,et al.  Synthetic brain imaging: grasping, mirror neurons and imitation , 2000, Neural Networks.

[10]  G. Palm,et al.  On associative memory , 2004, Biological Cybernetics.

[11]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[12]  W. M. Keck,et al.  Machine Psychology : Autonomous Behavior , Perceptual Categorization and Conditioning in a Brain-based Device , 2002 .

[13]  Günther Palm,et al.  Combining Visual Attention, Object Recognition and Associative Information Processing in a NeuroBotic System , 2005, Biomimetic Neural Learning for Intelligent Robots.