Emergent neural turing machine and its visual navigation

Traditional Turing Machines (TMs) are symbolic whose hand-crafted representations are static and limited. Developmental Network 1 (DN-1) uses emergent representation to perform Turing Computation. But DN-1 lacks hierarchy in its internal representations, and is hard to handle the complex visual navigation tasks. In this paper, we improve DN-1 with several new mechanisms and presenta new emergent neural Turing Machine - Developmental Network 2 (DN-2). By neural, we mean that the control of the TM has neurons as basic computing elements. The major novelty of DN-2 over DN-1 is that the representational hierarchy inside DN-2 is emergent and fluid. DN-2 grows complex hierarchies by dynamically allowing initialization of neurons with different ranges of connection. We present a complex task - vision guided navigation in simulated and natural worlds using DN-2. A major function that has not been demonstrated before is that the hierarchy enables attention that disregards distracting features based on the navigation context. In simulated navigation experiments, DN-2 can perform with no errors, and in real-world navigation experiments, the error rate is only 0.78%. These experimental results showed that DN-2 successfully learned rules of navigation with image inputs.

[1]  David Barber,et al.  Bayesian reasoning and machine learning , 2012 .

[2]  Jean Piaget,et al.  The Grasp of Consciousness (Psychology Revivals) : Action and Concept in the Young Child , 2015 .

[3]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[6]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[7]  Scott P. Johnson,et al.  Visual statistical learning in infancy: evidence for a domain general learning mechanism , 2002, Cognition.

[8]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[9]  Juyang Weng,et al.  Dually Optimal Neuronal Layers: Lobe Component Analysis , 2009, IEEE Transactions on Autonomous Mental Development.

[10]  Paul A. Gagniuc,et al.  Markov Chains: From Theory to Implementation and Experimentation , 2017 .

[11]  Marvin Minsky,et al.  Logical Versus Analogical or Symbolic Versus Connectionist or Neat Versus Scruffy , 1991, AI Mag..

[12]  Yoshua Bengio,et al.  Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes , 2016, ArXiv.

[13]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[14]  A. Turing.  On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[15]  L. Vygotsky.  Mind in Society: The Development of Higher Psychological Processes: Harvard University Press , 1978 .

[16]  Weiwei Huang,et al.  Cognitive memory and mapping in a brain-like system for robotic navigation , 2017, Neural Networks.

[17]  Eduardo Sontag,et al.  Turing computability with neural nets , 1991 .

[18]  Yoshiaki Shirai,et al.  Autonomous visual navigation of a mobile robot using a human-guided experience , 2002, Robotics Auton. Syst..

[19]  Juyang Weng,et al.  Brain as an Emergent Finite Automaton: A Theory and Three Theorems , 2015 .

[20]  Martin Llofriu,et al.  Goal-oriented robot navigation learning using a multi-scale space representation , 2015, Neural Networks.

[21]  Geoffrey E. Hinton.  Deep belief networks , 2009, Scholarpedia.

[22]  Anna Sarosiek.  Semantics and symbol grounding in Turing machine processes , 2018 .

[23]  Yoram Singer,et al.  The Hierarchical Hidden Markov Model: Analysis and Applications , 1998, Machine Learning.

[24]  Michael Sipser,et al.  Introduction to the Theory of Computation , 1996, SIGA.