Dynamic Cortex Memory Networks

is investigated. Hereby, DCM networks are analyzed regarding particular key features of neural signal processing systems, namely, their robustness to noise and their ability of time warping. Throughout all experiments we show that DCMs converge faster and yield better results than LSTMs. Hereby, DCM networks require overall less weights than pure LSTM networks to achieve the same or even better results. Besides, a promising neurally implemented just-in-time online signal filter approach is presented, which is latency-free and still provides an accurate filtering performance much better than conventional low-pass filters. We also show that the neural networks can do explicit time warping even better than the Dynamic Time Warp­ ing (DTW) algorithm, which is a specialized method developed for this task.

[1]  Andreas Zell,et al.  Dynamic Cortex Memory: Enhancing Recurrent Neural Networks for Gradient-Based Sequence Learning , 2014, ICANN.

[2]  Jürgen Schmidhuber,et al.  RNN-based Learning of Compact Maps for Efficient Robot Localization , 2007, ESANN.

[3]  Sebastian Otte,et al.  OCT A-Scan based lung tumor tissue classification with Bidirectional Long Short Term Memory networks , 2013, 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[4]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[5]  Philip Chan,et al.  Toward accurate dynamic time warping in linear time and space , 2007, Intell. Data Anal..

[6]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[7]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[8]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[9]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[10]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[11]  Marcus Liwicki,et al.  A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks , 2007 .

[12]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[13]  Sebastian Otte,et al.  JANNLab Neural Network Framework for Java , 2013, MLDM Posters.

[14]  Navdeep Jaitly,et al.  Hybrid speech recognition with Deep Bidirectional LSTM , 2013, 2013 IEEE Workshop on Automatic Speech Recognition and Understanding.

[15]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Jürgen Schmidhuber,et al.  Learning Precise Timing with LSTM Recurrent Networks , 2003, J. Mach. Learn. Res..

[17]  Sebastian Otte,et al.  Local Feature Based Online Mode Detection with Recurrent Neural Networks , 2012, 2012 International Conference on Frontiers in Handwriting Recognition.