Derivative-based imaging

We present a new biologically inspired imaging framework which uses sensor motion in a constructive way, contrary to most current imaging techniques. In addition to measuring light intensity, the spatial derivatives of the scene are estimated by measuring the temporal derivatives of the output of the vibrating photoreceptors. This is inspired from the fixational eye movements of the human visual system. When motion is known, these measured spatial derivatives can be used to improve the quality of the intensity image, as well as for object and pattern recognition tasks. This image reconstruction is based on integrating the derivatives while using the measured intensities as integration constants. The results of the imaging framework and of the reconstruction step are high quality images of the light intensities and of its spatial derivatives, which is relevant for further scene understanding. The advantages and the potential of this new imaging framework are shown in many simulations in this paper.

[1]  D. Hubel.  Eye, brain, and vision , 1988 .

[2]  Martina Poletti,et al.  Miniature eye movements enhance fine spatial detail , 2007, Nature.

[3]  Giacomo Indiveri,et al.  Analog VLSI architectures for motion processing: from fundamental limits to system applications , 1996, Proc. IEEE.

[4]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[5]  Rahul Sarpeshkar,et al.  An analog VLSI velocity sensor , 1995, Proceedings of ISCAS'95 - International Symposium on Circuits and Systems.

[6]  Ania Mitros,et al.  Visual Sensor with Resolution Enhancement by Mechanical Vibrations , 2001, Auton. Robots.

[7]  Maria Petrou,et al.  1 – Current trends in super-resolution image reconstruction , 2008 .

[8]  Jayanthi Sivaswamy,et al.  Hexagonal Image Processing: A Practical Approach , 2014, Advances in Pattern Recognition.

[9]  Peter I. Corke,et al.  Vibration isolation for autonomous helicopter flight , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[10]  D. Hubel,et al.  The role of fixational eye movements in visual perception , 2004, Nature Reviews Neuroscience.

[11]  Alireza Moini,et al.  Motion Perception Using Analog VLSI , 1998 .

[12]  Max-Olivier Hongler,et al.  The Resonant Retina: Exploiting Vibration Noise to Optimally Detect Edges in an Image , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  E. Culurciello,et al.  A biomorphic digital image sensor , 2003, IEEE J. Solid State Circuits.

[14]  Jan Van der Spiegel,et al.  A focal plane visual motion measurement sensor , 1997 .

[15]  Jang-Kyoo Shin,et al.  A foveated-structure CMOS retina chip for edge detection with local light adaptation , 2003 .

[16]  Jörg Kramer,et al.  An integrated optical transient sensor , 2002 .

[17]  Maria Petrou,et al.  Reverse engineering the human vision system: a possible explanation for the role of microsaccades , 2004, ICPR 2004.

[18]  Fumiya Iida,et al.  Biologically inspired visual odometer for navigation of a flying robot , 2003, Robotics Auton. Syst..

[19]  Tania Stathaki,et al.  Image Fusion: Algorithms and Applications , 2008 .

[20]  Thierry M. Bernard,et al.  Second generation programmable artificial retina , 1999, Twelfth Annual IEEE International ASIC/SOC Conference (Cat. No.99TH8454).

[21]  Andreas G. Andreou,et al.  A Contrast Sensitive Silicon Retina with Reciprocal Synapses , 1991, NIPS.

[22]  Tadashi Shibata,et al.  A motion-based analog VLSI saliency detector using quasi-two-dimensional hardware algorithm , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[23]  Paul R. Cooper,et al.  The dynamic retina: Contrast and motion detection for active vision , 2005, International Journal of Computer Vision.

[24]  Charles M. Higgins,et al.  Pulse-based 2-D motion sensors , 1999 .

[25]  Jayanthi Sivaswamy,et al.  Hexagonal Image Processing: A Practical Approach (Advances in Pattern Recognition) , 2005 .

[26]  Giulio Sandini,et al.  A Foveated Retina-Like Sensor Using CCD Technology , 1989, Analog VLSI Implementation of Neural Systems.

[27]  Carver A. Mead,et al.  Time-derivative adaptive silicon photoreceptor array , 1991, Optics & Photonics.

[28]  I. Shimoyama,et al.  Design and performance of a micro-sized biomorphic compound eye with a scanning retina , 2000, Journal of Microelectromechanical Systems.

[29]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[30]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[31]  T. Delbruck.  Silicon retina with correlation-based, velocity-tuned pixels , 1993 .

[32]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[33]  Christof Koch,et al.  An analog VLSI velocity sensor using the gradient method , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[34]  Derek Abbott,et al.  An analog implementation of early visual processing in insects , 1993, 1993 International Symposium on VLSI Technology, Systems, and Applications Proceedings of Technical Papers.

[35]  Alireza Moini,et al.  Vision Chips , 1999 .

[36]  Tobi Delbrück,et al.  A 128 X 128 120db 30mw asynchronous vision sensor that responds to relative intensity change , 2006, 2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers.

[37]  F. Barth,et al.  Sensors and Sensing in Biology and Engineering , 2003, Springer Vienna.

[38]  Giacomo Indiveri,et al.  Neuromorphic vision sensors and preprocessors in system applications , 1998, Other Conferences.

[39]  Dominique Ginhac,et al.  Design of a 10000 Frames/s CMOS Sensor with In Situ Image Processing , 2006, ReCoSoC.

[40]  Kazuo Kyuma,et al.  A 128×128-Pixel Artificial Retina LSI with Two-Dimensional Filtering Functions , 1999 .

[41]  Reid R. Harrison,et al.  A CMOS Imager with On-Chip Temporal Filtering for Motion Pre-Processing , 2002 .

[42]  Rahul Sarpeshkar,et al.  Pulse-Based Analog VLSI Velocity Sensors , 1997 .

[43]  Aggelos K. Katsaggelos,et al.  Super Resolution of Images and Video , 2006, Super Resolution of Images and Video.

[44]  R. Etienne-Cummings,et al.  A foveated silicon retina for two-dimensional tracking , 2000 .

[45]  C.A.T. Salama,et al.  Image-motion detection using analog VLSI , 1992 .

[46]  Jörg Kramer.  Compact Integrated Motion Sensor With Three-Pixel Interaction , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Alan A. Stocker,et al.  Compact integrated transconductance amplifier circuit for temporal differentiation , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[48]  Gilles Sicard,et al.  An adaptive bio–inspired analog silicon retina , 1999 .

[49]  Gert Cauwenberghs,et al.  Micropower gradient flow acoustic localizer , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[50]  Luigi Raffo,et al.  A Low-Power Integrated Smart Sensor with on-Chip Real-Time Image Processing Capabilities , 2005, EURASIP J. Adv. Signal Process..

[51]  Yitzhak Yitzhaky,et al.  Restoration of an image degraded by vibrations using only a single frame , 2000 .

[52]  Maria Petrou.  A new imaging architecture and an alternative interpretation of the structure of the human retina , 2008, 2008 International Symposium on Information Technology.

[53]  Giulio Sandini,et al.  Retina-Like Sensors: Motivations, Technology and Applications , 2003 .

[54]  Wesley E. Snyder,et al.  Coordinate system for hexagonal pixels , 1999, Medical Imaging.

[55]  Stéphane Viollet,et al.  Bio-inspired optical flow circuits for the visual guidance of micro air vehicles , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..