Bandwidth Modeling of Silicon Retinas for Next Generation Visual Sensor Networks

Silicon retinas, also known as Dynamic Vision Sensors (DVS) or event-based visual sensors, have shown great advantages in terms of low power consumption, low bandwidth, wide dynamic range and very high temporal resolution. Owing to such advantages as compared to conventional vision sensors, DVS devices are gaining more and more attention in various applications such as drone surveillance, robotics, high-speed motion photography, etc. The output of such sensors is a sequence of events rather than a series of frames as for classical cameras. Estimating the data rate of the stream of events associated with such sensors is needed for the appropriate design of transmission systems involving such sensors. In this work, we propose to consider information about the scene content and sensor speed to support such estimation, and we identify suitable metrics to quantify the complexity of the scene for this purpose. According to the results of this study, the event rate shows an exponential relationship with the metric associated with the complexity of the scene and linear relationships with the speed of the sensor. Based on these results, we propose a two-parameter model for the dependency of the event rate on scene complexity and sensor speed. The model achieves a prediction accuracy of approximately 88.4% for the outdoor environment along with the overall prediction performance of approximately 84%.

[1]  Misha Mahowald,et al.  A silicon model of early visual processing , 1993, Neural Networks.

[2]  Maria G. Martini,et al.  H.264/MPEG-AVC, H.265/MPEG-HEVC and VP9 codec comparison for live gaming video streaming , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[3]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[4]  Tobi Delbrück,et al.  The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM , 2016, Int. J. Robotics Res..

[5]  Narciso García,et al.  Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  P. Machado,et al.  Using complexity estimates in aesthetic image classification , 2012 .

[7]  Jim R. Parker,et al.  Algorithms for image processing and computer vision , 1996 .

[8]  Dirk Staehle,et al.  Opportunistic QoS-Aware Fair Downlink Scheduling for Delay Sensitive Applications Using Fuzzy Reactive and Proactive Controllers , 2013, 2013 IEEE 78th Vehicular Technology Conference (VTC Fall).

[9]  Juan Barrios-Aviles,et al.  Less Data Same Information for Event-Based Sensors: A Bioinspired Filtering and Data Reduction Algorithm , 2018, Sensors.

[10]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Lawrence G. Roberts,et al.  Machine Perception of Three-Dimensional Solids , 1963, Outstanding Dissertations in the Computer Sciences.

[12]  Massimo A. Sivilotti,et al.  Wiring considerations in analog VLSI systems, with application to field-programmable networks , 1992 .

[13]  Paul M. B. Vitányi,et al.  Clustering by compression , 2003, IEEE Transactions on Information Theory.

[14]  Davide Scaramuzza,et al.  Feature detection and tracking with the dynamic and active-pixel vision sensor (DAVIS) , 2016, 2016 Second International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP).

[15]  Fariborz Baghaei Naeini,et al.  A Novel Event-Based Incipient Slip Detection Using Dynamic Active-Pixel Vision Sensor (DAVIS) , 2018, Sensors.

[16]  Sebastian Möller,et al.  A Comparative Quality Assessment Study for Gaming and Non-Gaming Videos , 2018, 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).

[17]  Yuri Shelepin,et al.  Complexity of Images: Experimental and Computational Estimates Compared , 2012, Perception.

[18]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[19]  T. Delbruck,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < 1 , 2022 .

[20]  Tobi Delbrück,et al.  A 128 X 128 120db 30mw asynchronous vision sensor that responds to relative intensity change , 2006, 2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers.

[21]  S. Nagata,et al.  An electronic model of the retina , 1970 .

[22]  Yvonne Koch Algorithms For Image Processing And Computer Vision , 2016 .

[23]  Dirk Staehle,et al.  QoS-aware composite scheduling using fuzzy proactive and reactive controllers , 2014, EURASIP J. Wirel. Commun. Netw..

[24]  Hong-Hsu Yen,et al.  A Survey on Sensor Coverage and Visual Data Capturing/Processing/Transmission in Wireless Visual Sensor Networks , 2014, Sensors.

[25]  Maria G. Martini,et al.  Data rate estimation based on scene complexity for dynamic vision sensors on unmanned vehicles , 2018, 2018 IEEE 29th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC).

[26]  Maria G. Martini,et al.  Content-aware downlink scheduling for LTE wireless systems: A survey and performance comparison of key approaches , 2018, Comput. Commun..

[27]  A. Haseeb,et al.  Rate and distortion modeling for real-time MGS coding and adaptation , 2012, 2012 Wireless Advanced (WiAd).

[28]  Tiejun Huang,et al.  Spike Coding for Dynamic Vision Sensors , 2018, 2018 Data Compression Conference.

[29]  Maria G. Martini,et al.  QoE-driven multi-user scheduling and rate adaptation with reduced cross-layer signaling for scalable video streaming over LTE wireless systems , 2016, EURASIP J. Wirel. Commun. Netw..

[30]  Davide Scaramuzza,et al.  Event-based, 6-DOF pose tracking for high-speed maneuvers , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]  Tobi Delbruck,et al.  A 240 × 180 130 dB 3 µs Latency Global Shutter Spatiotemporal Vision Sensor , 2014, IEEE Journal of Solid-State Circuits.

[32]  Ryad Benosman,et al.  Giving machines humanlike eyes , 2015, IEEE Spectrum.

[33]  Vito Di Gesù,et al.  A fuzzy approach to the evaluation of image complexity , 2009, Fuzzy Sets Syst..

[34]  Davide Scaramuzza,et al.  Lifetime estimation of events from Dynamic Vision Sensors , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Stefan Winkler,et al.  Image complexity and spatial information , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[36]  Bin Ma,et al.  The similarity metric , 2001, IEEE Transactions on Information Theory.