A $64\times64$ Pixel Vision Sensor for Local Binary Pattern Computation

We report on a <inline-formula> <tex-math notation="LaTeX">$64\times64$ </tex-math></inline-formula> pixel vision sensor embedding local binary pattern (LBP) computation implemented at pixel-level. The proposed technique is applied on four neighbours within a <inline-formula> <tex-math notation="LaTeX">$3\times3$ </tex-math></inline-formula> pixel kernel, while the operations are executed on-the-fly, during the sensor exposure time. Thanks to the pixel-level autoexposure control, processing over extended dynamic range is achieved. The number of transistors per pixel has been minimized in order to preserve the fill factor while keeping a reduced pixel size. The resulting binary information is stored on a dynamic memory and read out in a standard raster-scan mode. The proposed technique for the LBP estimation is made in the time domain and uses two voltage comparators per pixel. While the chronological order of comparators toggling identifies the brighter pixel, the voltage difference between the two thresholds sets the LBP sensitivity. The sensor is implemented in a CMOS <inline-formula> <tex-math notation="LaTeX">$0.35~\mu \text{m}$ </tex-math></inline-formula>, featuring a 17-<inline-formula> <tex-math notation="LaTeX">$\mu \text{m}$ </tex-math></inline-formula> pitch with 15% fill factor. The chip consumes <inline-formula> <tex-math notation="LaTeX">$35~\mu \text{W}$ </tex-math></inline-formula> at 15 frames/s, powered at 3.3 V for the analog part and 1.5 V for the digital part.

[1]  Aiping Feng,et al.  Simplified Local Binary Pattern Descriptor for Character Recognition of Vehicle License Plate , 2010, 2010 Seventh International Conference on Computer Graphics, Imaging and Visualization.

[2]  Youdong Ding,et al.  Recognition of hand-gestures using improved local binary pattern , 2011, 2011 International Conference on Multimedia Technology.

[3]  Michela Lecca,et al.  On the von Kries Model: Estimation, Dependence on Light and Device, and Applications , 2014 .

[4]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Bernabé Linares-Barranco,et al.  A Spatial Contrast Retina With On-Chip Calibration for Neuromorphic Spike-Based AER Vision Systems , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[6]  R. Etienne-Cummings,et al.  Temporal change threshold detection imager , 2005, ISSCC. 2005 IEEE International Digest of Technical Papers. Solid-State Circuits Conference, 2005..

[7]  Mahdi Jampour,et al.  Efficient Handwritten Digit Recognition based on Histogram of Oriented Gradients and SVM , 2014 .

[8]  Matti Pietikäinen,et al.  Rotation-Invariant Image and Video Description With Local Binary Pattern Features , 2012, IEEE Transactions on Image Processing.

[9]  M. Gottardi,et al.  A 100 $\mu$ W 128 $\times$ 64 Pixels Contrast-Based Asynchronous Binary Vision Sensor for Sensor Networks Applications , 2009, IEEE Journal of Solid-State Circuits.

[10]  Caifeng Shan,et al.  Learning local binary patterns for gender classification on real-world face images , 2012, Pattern Recognit. Lett..

[11]  Ioannis Pratikakis,et al.  A two-stage scheme for text detection in video images , 2010, Image Vis. Comput..

[12]  Massimo Gottardi,et al.  A 30 µW 30 fps 110 × 110 Pixels Vision Sensor Embedding Local Binary Patterns , 2015, IEEE Journal of Solid-State Circuits.

[13]  Nasser Kehtarnavaz,et al.  Action Recognition from Depth Sequences Using Depth Motion Maps-Based Local Binary Patterns , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[14]  Shuicheng Yan,et al.  Discriminative local binary patterns for human detection in personal album , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Massimo Gottardi,et al.  A 35μW 64 × 64 Pixels Vision Sensor Embedding Local Binary Pattern Code Computation , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[16]  M. Pietikäinen,et al.  TEXTURE ANALYSIS WITH LOCAL BINARY PATTERNS , 2004 .

[17]  Matti Pietikäinen,et al.  Block-Based Methods for Image Retrieval Using Local Binary Patterns , 2005, SCIA.

[18]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[19]  Yiding Wang,et al.  Hand-dorsa vein recognition based on partition Local Binary Pattern , 2010, IEEE 10th INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING PROCEEDINGS.

[20]  Jaehyuk Choi,et al.  A 3.4-$\mu$ W Object-Adaptive CMOS Image Sensor With Embedded Feature Extraction Algorithm for Motion-Triggered Object-of-Interest Imaging , 2014, IEEE Journal of Solid-State Circuits.

[21]  Matti Pietikäinen,et al.  Texture classification by center-symmetric auto-correlation, using Kullback discrimination of distributions , 1995, Pattern Recognit. Lett..

[22]  Sébastien Marcel,et al.  Local binary patterns as an image preprocessing for face authentication , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[23]  Kang Ryoung Park,et al.  Finger vein recognition using minutia‐based alignment and local binary pattern‐based feature extraction , 2009, Int. J. Imaging Syst. Technol..

[24]  Amine Bermak,et al.  A 2PJ/Pixel/Direction MIMO Processing Based CMOS Image Sensor for Omnidirectional Local Binary Pattern Extraction and Edge Detection , 2018, 2018 IEEE Symposium on VLSI Circuits.

[25]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Di Huang,et al.  Local Binary Patterns and Its Application to Facial Image Analysis: A Survey , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).