Probabilistic Undirected Graph Based Denoising Method for Dynamic Vision Sensor

Dynamic Vision Sensor (DVS) is a new type of neuromorphic event-based sensor, which has an innate advantage in capturing fast-moving objects. Due to the interference of DVS hardware itself and many external factors, noise is unavoidable in the output of DVS. Different from frame/image with structural data, the output of DVS is in the form of addressevent representation (AER), which means that the traditional denoising methods cannot be used for the output (i.e., event stream) of the DVS. In this paper, we propose a novel event stream denoising method based on probabilistic undirected graph model (PUGM). The motion of objects always shows a certain regularity/trajectory in space and time, which reflects the spatiotemporal correlation between effective events in the stream. Meanwhile, the event stream of DVS is composed by the effective events and random noise. Thus, a probabilistic undirected graph model is constructed to describe such priori knowledge (i.e., spatio-temporal correlation). The undirected graph model is factorized into the product of the cliques energy function, and the energy function is defined to obtain the complete expression of the joint probability distribution. Better denoising effect means a higher probability (lower energy), which means the denoising problem can be transfered into energy optimization problem. Thus, the iterated conditional modes (ICM) algorithm is used to optimize the model to remove the noise. Experimental results on denoising show that the proposed algorithm can effectively remove noise events. Moreover, with the preprocessing of the proposed algorithm, the recognition accuracy on AER data can be remarkably promoted. The source code of the proposed method is available at https://web.xidian.edu.cn/wjj/paper.html.

[1]  Jean-Michel Morel,et al.  Towards a Bayesian Video Denoising Method , 2015, ACIVS.

[2]  Eric R. Fossum,et al.  CMOS image sensors: electronic camera on a chip , 1995, Proceedings of International Electron Devices Meeting.

[3]  Guangming Shi,et al.  An Improved Approach for Visualizing Dynamic Vision Sensor and its Video Denoising , 2017, ICVIP.

[4]  T. Delbruck,et al.  A 128 128 120 dB 15 s Latency Asynchronous Temporal Contrast Vision Sensor , 2006 .

[5]  Li-Wei Kang,et al.  Self-Learning Based Image Decomposition With Applications to Single Image Denoising , 2014, IEEE Transactions on Multimedia.

[6]  Guangming Shi,et al.  DVS image noise removal using K-SVD method , 2018, International Conference on Graphic and Image Processing.

[7]  Li Xia,et al.  Event-based sensor activation for indoor occupant distribution estimation , 2012, 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV).

[8]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[9]  Shoushun Chen,et al.  Dynamic resolution event-based temporal contrast vision sensor , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[10]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[11]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[12]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Karen O. Egiazarian,et al.  Video denoising using separable 4D nonlocal spatiotemporal transforms , 2011, Electronic Imaging.

[14]  A. Bevilacqua,et al.  An ultrawideband CMOS low-noise amplifier for 3.1-10.6-GHz wireless receivers , 2004, IEEE Journal of Solid-State Circuits.

[15]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[16]  Stefan Leutenegger,et al.  Simultaneous Optical Flow and Intensity Estimation from an Event Camera , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  William T. Freeman,et al.  What makes a good model of natural images? , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Bernabé Linares-Barranco,et al.  Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[19]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[20]  Bernabé Linares-Barranco,et al.  A 128$\,\times$ 128 1.5% Contrast Sensitivity 0.9% FPN 3 µs Latency 4 mW Asynchronous Frame-Free Dynamic Vision Sensor Using Transimpedance Preamplifiers , 2013, IEEE Journal of Solid-State Circuits.

[21]  Stan Z. Li,et al.  Markov Random Field Modeling in Image Analysis , 2001, Computer Science Workbench.

[22]  Shi Yan,et al.  Effective target binarization method for linear timed address-event vision system , 2016 .

[23]  Michael J. Black,et al.  Efficient Belief Propagation with Learned Higher-Order Markov Random Fields , 2006, ECCV.

[24]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[25]  Jean-Michel Morel,et al.  Video Denoising via Empirical Bayesian Estimation of Space-Time Patches , 2017, Journal of Mathematical Imaging and Vision.

[26]  Patrick Bouthemy,et al.  Space-Time Adaptation for Patch-Based Image Sequence Restoration , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Chiara Bartolozzi,et al.  Event-Based Vision: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Bo Zhao,et al.  Bag of Events: An Efficient Probability-Based Feature Extraction Method for AER Image Sensors , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Bernabé Linares-Barranco,et al.  Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  David Zhang,et al.  Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Kjell O. Jeppson,et al.  CMOS Circuit Speed and Buffer Optimization , 1987, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[32]  Rogério Schmidt Feris,et al.  Joint Super Resolution and Denoising From a Single Depth Image , 2015, IEEE Transactions on Multimedia.