Efficient high speed signal estimation with neuromorphic vision sensors

Recently developed neuromorphic vision sensors present a high speed, event-based alternative to conventional vision in robotic systems. We present a method for the design of simple, low computation estimators that take advantage of the remarkable properties of these sensors to provide low latency, high bandwidth sensing for feedback control of fast dynamical systems. It is shown that under certain circumstances a simple transformation of the event stream from such a sensor can allow it to be treated as an asynchronous configuration sensor, with minimal computation required to achieve high speed signal reconstruction. These results are applicable to any robotic control problems requiring high performance visual feedback control with low computation.

[1]  Emilio Frazzoli,et al.  A Power-Performance Approach to Comparing Sensor Families, with application to comparing neuromorphic to traditional vision sensors , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Davide Scaramuzza,et al.  Low-latency event-based visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Christian Stöcker,et al.  Event-Based Control , 2014 .

[4]  Tobi Delbruck,et al.  Robotic goalie with 3 ms reaction time at 4% CPU load using event-based dynamic vision sensor , 2013, Front. Neurosci..

[5]  Tobi Delbrück,et al.  Low-latency localization by active LED markers tracking using a dynamic vision sensor , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Steve B. Furber,et al.  Real-Time Interface Board for Closed-Loop Robotic Tasks on the SpiNNaker Neural Computing System , 2013, ICANN.

[7]  Bernabé Linares-Barranco,et al.  A 128$\,\times$ 128 1.5% Contrast Sensitivity 0.9% FPN 3 µs Latency 4 mW Asynchronous Frame-Free Dynamic Vision Sensor Using Transimpedance Preamplifiers , 2013, IEEE Journal of Solid-State Circuits.

[8]  Ryad Benosman,et al.  Asynchronous Event-Based Visual Shape Tracking for Stable Haptic Feedback in Microrobotics , 2012, IEEE Transactions on Robotics.

[9]  Jörg Conradt,et al.  Self-calibrating Marker Tracking in 3D with Event-Based Vision Sensors , 2012, ICANN.

[10]  Chiara Bartolozzi,et al.  Asynchronous frameless event-based optical flow , 2012, Neural Networks.

[11]  Tobi Delbrück,et al.  Asynchronous Event-Based Binocular Stereo Matching , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Bernabé Linares-Barranco,et al.  A 3.6 $\mu$ s Latency Asynchronous Frame-Free Event-Driven Dynamic-Vision-Sensor , 2011, IEEE Journal of Solid-State Circuits.

[13]  Stephan Schraml,et al.  A spatio-temporal clustering method using real-time motion analysis on event-based 3D vision , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[14]  Daniel Matolin,et al.  A QVGA 143dB dynamic range asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression , 2010, 2010 IEEE International Solid-State Circuits Conference - (ISSCC).

[15]  Jan Lunze,et al.  A state-feedback approach to event-based control , 2010, Autom..

[16]  Tobi Delbrück,et al.  A pencil balancing robot using a pair of AER dynamic vision sensors , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[17]  Tobias Delbrück,et al.  Frame-free dynamic digital vision , 2008 .

[18]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[19]  Karl Johan Åström,et al.  Event Based Control , 2008 .

[20]  Robin Ritz.  Development of a Dynamical Model for Retina-based Control of an RC Monster Truck , 2008 .

[21]  Karl Johan Åström,et al.  On limit cycles in event-based control systems , 2007, 2007 46th IEEE Conference on Decision and Control.

[22]  Martin Litzenberger,et al.  Embedded Smart Camera for High Speed Vision , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[23]  Paulo Tabuada,et al.  Event-Triggered Real-Time Scheduling of Stabilizing Control Tasks , 2007, IEEE Transactions on Automatic Control.

[24]  T. Delbruck,et al.  A 128 128 120 dB 15 s Latency Asynchronous Temporal Contrast Vision Sensor , 2006 .

[25]  Tobi Delbrück,et al.  AER Building Blocks for Multi-Layer Multi-Chip Neuromorphic Vision Systems , 2005, NIPS.

[26]  Peter Corke,et al.  VISUAL CONTROL OF ROBOT MANIPULATORS – A REVIEW , 1993 .

[27]  Peter Corke.  High-performance visual servoing for robot end-point control , 1993, Other Conferences.

[28]  Patrick Rives,et al.  A new approach to visual servoing in robotics , 1992, IEEE Trans. Robotics Autom..

[29]  Patrick Rives,et al.  Positioning of a robot with respect to an object, tracking it and estimating its velocity by visual servoing , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.