A 30 µW 30 fps 110 × 110 Pixels Vision Sensor Embedding Local Binary Patterns

We present a 110 × 110 pixel vision sensor that computes the Local Binary Patterns (LBPs) of an imaged scene with a power consumption of 30 μW at 30 fps. The LBP of a given pixel is a binary vector, encoding the direction and sign of image contrast with respect to its neighbors. Each LBP provides a visual description of an image's local structure that is widely used for texture and object recognition. In the sensor proposed here, each pixel detects its corresponding LBP with respect to its four neighboring pixels and saves this information into a digital map using 6 bits to encode each pixel. The operation is executed during the exposure time and requires 83 pW/pixel · frame to be computed. The chip is implemented in a 0.35 μm CMOS featuring 34 T square pixels with 26 μm pitch. We illustrate some examples of image description based on the LBPs output by the sensor.

[1]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Sanshiro Shishido,et al.  A 3.6pW/frame·pixel 1.35V PWM CMOS Imager with Dynamic Pixel Readout and no Static Bias Current , 2008, 2008 IEEE International Solid-State Circuits Conference - Digest of Technical Papers.

[3]  Jaehyuk Choi,et al.  A 3.4-$\mu$ W Object-Adaptive CMOS Image Sensor With Embedded Feature Extraction Algorithm for Motion-Triggered Object-of-Interest Imaging , 2014, IEEE Journal of Solid-State Circuits.

[4]  Di Huang,et al.  Local Binary Patterns and Its Application to Facial Image Analysis: A Survey , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[5]  Matti Pietikäinen,et al.  Texture classification by center-symmetric auto-correlation, using Kullback discrimination of distributions , 1995, Pattern Recognit. Lett..

[6]  Marko Heikkilä,et al.  A texture-based method for modeling the background and detecting moving objects , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  E. Culurciello,et al.  A biomorphic digital image sensor , 2003, IEEE J. Solid State Circuits.

[8]  Tobi Delbrück,et al.  A 128 X 128 120db 30mw asynchronous vision sensor that responds to relative intensity change , 2006, 2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers.

[9]  E. Culurciello,et al.  A 1-mW CMOS Temporal-Difference AER Sensor for Wireless Sensor Networks , 2009, IEEE Transactions on Electron Devices.

[10]  Mark D. Fairchild,et al.  The HDR Photographic Survey , 2007, CIC.

[11]  David Blaauw,et al.  A 467nW CMOS visual motion sensor with temporal averaging and pixel aggregation , 2013, 2013 IEEE International Solid-State Circuits Conference Digest of Technical Papers.

[12]  L. Gonzo,et al.  Novel CMOS image sensor with a 132-dB dynamic range , 2002, IEEE J. Solid State Circuits.

[13]  Loris Nanni,et al.  Survey on LBP based texture descriptors for image classification , 2012, Expert Syst. Appl..

[14]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Bernabé Linares-Barranco,et al.  A Spatial Contrast Retina With On-Chip Calibration for Neuromorphic Spike-Based AER Vision Systems , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[16]  R. Etienne-Cummings,et al.  Temporal change threshold detection imager , 2005, ISSCC. 2005 IEEE International Digest of Technical Papers. Solid-State Circuits Conference, 2005..

[17]  Loris Nanni,et al.  Local binary patterns for a hybrid fingerprint matcher , 2008, Pattern Recognit..

[18]  Matti Pietikäinen,et al.  Rotation-invariant texture classification using feature distributions , 2000, Pattern Recognit..

[19]  Jaehyuk Choi,et al.  A 3.4µW CMOS image sensor with embedded feature-extraction algorithm for motion-triggered object-of-interest imaging , 2013, 2013 IEEE International Solid-State Circuits Conference Digest of Technical Papers.

[20]  M. Gottardi,et al.  A 33 $\mu$ W 64$\,\times\,$ 64 Pixel Vision Sensor Embedding Robust Dynamic Background Subtraction for Event Detection and Scene Interpretation , 2013, IEEE Journal of Solid-State Circuits.

[21]  M. Gottardi,et al.  A 100 $\mu$ W 128 $\times$ 64 Pixels Contrast-Based Asynchronous Binary Vision Sensor for Sensor Networks Applications , 2009, IEEE Journal of Solid-State Circuits.

[22]  Eric R. Fossum,et al.  A 1.5-V 550-/spl mu/W 176/spl times/144 autonomous CMOS active pixel image sensor , 2003 .