On-Array Compressive Acquisition in CMOS Image Sensors Using Accumulated Spatial Gradients

A compressive acquisition technique for on-array image compression is proposed in this paper. It capitalizes on representation ability of accumulated spatial gradients of the acquired scene. The local variations inferred from strength of the accumulated gradients are used as cues to vary number of samples read through the image sensor readout. Such sampling enables the reconstruction using traditional interpolation techniques with desired quality. The proposed method is first verified using MATLAB simulations, where on an average, a compression of 87% is achieved, for a threshold of 40 intensity levels. The images are reconstructed using nearest neighbour interpolation (NNI) method which results in a mean peak signal to noise ratio (PSNR) value of 29.09 dB. The reconstructed images are further enhanced using deep convolutional neural network, which improves the PSNR to 32.46 dB. The biggest advantage of the proposed technique is low-complex hardware design. As a proof of concept, a hardware implementation of the technique is performed using discrete components. Pixel intensity values of standard images are converted into analog voltages using a data acquisition system and mapped in the input voltage range of 1.5 V −5.5 V. For a threshold of 3.8 V, the compression of 81% - 83% is observed for the considered images. The proposed technique is simple and effective, and is suitable for low-power complementary metal oxide semiconductor (CMOS) image sensors.

[1]  Soon-Jyh Chang,et al.  Low-Power and Wide-Bandwidth Cyclic ADC With Capacitor and Opamp Reuse Techniques for CMOS Image Sensor Application , 2009, IEEE Sensors Journal.

[2]  H. Ohtake,et al.  A 33-Megapixel 120-Frames-Per-Second 2.5-Watt CMOS Image Sensor With Column-Parallel Two-Stage Cyclic Analog-to-Digital Converters , 2012, IEEE Transactions on Electron Devices.

[3]  Siddhant Jain,et al.  Content Driven On-Chip Compression and Time Efficient Reconstruction for Image Sensor Applications , 2018, IEEE Sensors Journal.

[4]  Namil Kim,et al.  Multispectral pedestrian detection: Benchmark dataset and baseline , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[6]  Shoji Kawahito,et al.  A High-Speed CMOS Image Sensor with On-chip Parallel Image Compression Circuits , 2007, 2007 IEEE Custom Integrated Circuits Conference.

[7]  Abbas El Gamal,et al.  CMOS Image Sensor With Per-Column ΣΔ ADC and Programmable Compressed Sensing , 2013, IEEE Journal of Solid-State Circuits.

[8]  Shuai Wan,et al.  Adaptive Sampling for Image Compressed Sensing Based on Deep Learning , 2019 .

[9]  Farokh Marvasti,et al.  Adaptive Sparse Image Sampling and Recovery , 2017, IEEE Transactions on Computational Imaging.

[10]  Teresa H. Y. Meng,et al.  Transpose memory for video rate JPEG compression on highly parallel single-chip digital CMOS imager , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[11]  Santanu Chaudhury,et al.  Ultrasound Image Enhancement Using Structure Oriented Adversarial Network , 2018, IEEE Signal Processing Letters.

[12]  Amine Bermak,et al.  A Low-Power Pilot-DAC Based Column Parallel 8b SAR ADC With Forward Error Correction for CMOS Image Sensors , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[13]  Shoushun Chen,et al.  A Two-Step Prediction ADC Architecture for Integrated Low Power Image Sensors , 2017, IEEE Transactions on Circuits and Systems I: Regular Papers.

[14]  Mehrdad Soumekh.  Multiresolution dynamic image representation with uniform and foveal spiral scan data , 1998, IEEE Trans. Image Process..

[15]  Deepak Mishra,et al.  A super-pixel based on-chip image compression for high speed CMOS image sensors , 2017, 2017 International Conference on Electron Devices and Solid-State Circuits (EDSSC).

[16]  Amandeep Kaur,et al.  A low power low latency comparator for ramp ADC in CMOS imagers , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[17]  Amine Bermak,et al.  A Column-Parallel Inverter-Based Cyclic ADC for CMOS Image Sensor With Capacitance and Clock Scaling , 2016, IEEE Transactions on Electron Devices.

[18]  Aggelos K. Katsaggelos,et al.  Adaptive Image Sampling Using Deep Learning and Its Application on X-Ray Fluorescence Image Reconstruction , 2018, IEEE Transactions on Multimedia.

[19]  Amine Bermak,et al.  A Low-Power Compression-Based CMOS Image Sensor With Microshift-Guided SAR ADC , 2018, IEEE Transactions on Circuits and Systems II: Express Briefs.

[20]  Santanu Chaudhury,et al.  Ultrasound Image Segmentation: A Deeply Supervised Network With Attention to Boundaries , 2019, IEEE Transactions on Biomedical Engineering.

[21]  Jan Van der Spiegel,et al.  Design of a Digital Address-Event Triggered Compressive Acquisition Image Sensor , 2016, IEEE Transactions on Circuits and Systems I: Regular Papers.

[22]  Akira Matsuzawa,et al.  A CMOS image sensor with analog two-dimensional DCT-based compression circuits for one-chip cameras , 1997, IEEE J. Solid State Circuits.

[23]  Zheyao Wang,et al.  Data Compression for Image Sensor Arrays Using a 15-bit Two-Step Sigma–Delta ADC , 2014, IEEE Sensors Journal.

[24]  R. Genov,et al.  Focal-Plane Algorithmically-Multiplying CMOS Computational Image Sensor , 2009, IEEE Journal of Solid-State Circuits.

[25]  Michael W. Hoffman,et al.  A CMOS Imager With Focal Plane Compression Using Predictive Coding , 2007, IEEE Journal of Solid-State Circuits.

[26]  R. Sibson,et al.  A brief description of natural neighbor interpolation , 1981 .

[27]  André Kaup,et al.  Resampling Images to a Regular Grid From a Non-Regular Subset of Pixel Positions Using Frequency Selective Reconstruction , 2015, IEEE Transactions on Image Processing.

[28]  Shoji Kawahito,et al.  6.9 A 1.1µm 33Mpixel 240fps 3D-stacked CMOS image sensor with 3-stage cyclic-based analog-to-digital converters , 2016, 2016 IEEE International Solid-State Circuits Conference (ISSCC).

[29]  Amine Bermak,et al.  CMOS Image Sensor with On-Chip Image Compression: A Review and Performance Analysis , 2010, J. Sensors.

[30]  Deepak Mishra,et al.  A 12-bit, 2.5-bit/Phase Column-Parallel Cyclic ADC , 2019, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[31]  Anubha Gupta,et al.  Image Reconstruction Using Matched Wavelet Estimated From Data Sensed Compressively Using Partial Canonical Identity Matrix , 2017, IEEE Transactions on Image Processing.