Neuromorphic self-driving robot with retinomorphic vision and spike-based processing/closed-loop control

We present our work on a neuromorphic self-driving robot that employs retinomoprhic visual sensing and spike based processing. The robot senses the world through a spike-based visual system - the Asynchronous Time-based Image Sensor (ATIS) - and processes the sensory data stream using IBM's TrueNorth Neurosynaptic System. A convolutional neural network (CNN) running on the TrueNorth determines the steering direction based on what the ATIS “sees.” The network was trained on data from three different environments (indoor hallways, large campus sidewalks, and narrow neighborhood sidewalks) and achieved steering decision accuracies from 68% to 82% on development data from each dataset.

[1]  Andrew S. Cassidy,et al.  Real-Time Scalable Cortical Computing at 46 Giga-Synaptic OPS/Watt with ~100× Speedup in Time-to-Solution and ~100,000× Reduction in Energy-to-Solution , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[2]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[3]  Yann LeCun,et al.  Off-Road Obstacle Avoidance through End-to-End Learning , 2005, NIPS.

[4]  Daniel Matolin,et al.  A QVGA 143 dB Dynamic Range Frame-Free PWM Image Sensor With Lossless Pixel-Level Video Compression and Time-Domain CDS , 2011, IEEE Journal of Solid-State Circuits.

[5]  Andrew S. Cassidy,et al.  Convolutional networks for fast, energy-efficient neuromorphic computing , 2016, Proceedings of the National Academy of Sciences.

[6]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[7]  Andrew S. Cassidy,et al.  Cognitive computing programming paradigm: A Corelet Language for composing networks of neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[8]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[9]  Myron Flickner,et al.  Compass: A scalable simulator for an architecture for cognitive computing , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[10]  Andrew S. Cassidy,et al.  Design of silicon brains in the nano-CMOS era: Spiking neurons, learning synapses and neural architecture optimization , 2013, Neural Networks.

[11]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[12]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[13]  Jeffrey L. Krichmar,et al.  A self-driving robot using deep convolutional neural networks on neuromorphic hardware , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[14]  Andreas G. Andreou,et al.  Neuromorphic Engineering: From Neural Systems to Brain-Like Engineered Systems , 2013, Neural Networks.

[15]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.