An Algorithm Architecture Co-Design for CMOS Compressive High Dynamic Range Imaging

Standard image sensors feature dynamic range about 60 to 70 dB while the light flux of natural scenes may be over 120 dB. Most imagers dedicated to address such dynamic ranges, need specific, and large pixels. However, canonical imagers can be used for high dynamic range (HDR) by performing multicapture acquisitions to compensate saturation. This technique is made possible at the expense of the need for large memory requirements and an increase of the overall acquisition time. On the other hand, the implementation of compressive sensing (CS) raises the same issues regarding the modifications of both the pixel and the readout circuitry. Assuming HDR images are sufficiently sparse, CS claims they can be reconstructed from few random linear measurements. A novel CS-based image sensor design is presented in this paper allowing a compressive acquisition without changing the classical pixel design, as well as the overall sensor architecture. In addition to regular CS, HDR CS is enabled thanks to specific time diagrams of the control signals. An alternative nondestructive column-based readout mode constitutes the main change compared to a traditional functioning. The HDR reconstruction, which is also presented in this paper, is based on merging the information of multicapture compressed measurements while taking into account noise sources and nonlinearities introduced by both the proposed acquisition scheme and its practical implementation.

[1]  Michael W. Hoffman,et al.  A CMOS Image Sensor for Multi-Level Focal Plane Image Decomposition , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[2]  Ramesh Raskar,et al.  Unbounded High Dynamic Range Photography Using a Modulo Camera , 2015, 2015 IEEE International Conference on Computational Photography (ICCP).

[3]  M. Tchagaspanian,et al.  A SMALL FOOTPRINT, STREAMING COMPLIANT, VERSATILE WAVELET COMPRESSION SCHEME FOR CAMERAPHONE IMAGERS , 2009 .

[4]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[5]  B. Choubey,et al.  Low dark current logarithmic pixels , 2005, 48th Midwest Symposium on Circuits and Systems, 2005..

[7]  Pierre Vandergheynst,et al.  Compressed Sensing: “When Sparsity Meets Sampling” , 2011 .

[8]  Marcel J. M. Pelgrom,et al.  Matching properties of MOS transistors , 1989 .

[9]  Jan Van der Spiegel,et al.  A CMOS linear voltage/current dual-mode imager , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[10]  Michael W. Hoffman,et al.  A CMOS Imager With Focal Plane Compression Using Predictive Coding , 2007, IEEE Journal of Solid-State Circuits.

[11]  James E. Fowler,et al.  Block-Based Compressed Sensing of Images and Video , 2012, Found. Trends Signal Process..

[12]  Pierre Vandergheynst,et al.  UNLocBoX A matlab convex optimization toolbox using proximal splitting methods , 2014, ArXiv.

[13]  Shahram Shirani,et al.  Block-Based CS in a CMOS Image Sensor , 2014, IEEE Sensors Journal.

[14]  Pierre Vandergheynst,et al.  Power-efficient CMOS image acquisition system based on compressive sampling , 2013, 2013 IEEE 56th International Midwest Symposium on Circuits and Systems (MWSCAS).

[15]  Franco Maloberti,et al.  Analog Design for CMOS VLSI Systems , 2001 .

[16]  Eric R. Fossum,et al.  Digital camera system on a chip , 1998, IEEE Micro.

[17]  Abbas El Gamal,et al.  Quantitative study of high-dynamic-range image sensor architectures , 2004, IS&T/SPIE Electronic Imaging.

[18]  Pierre Vandergheynst,et al.  Multi-capture High Dynamic Range compressive imaging , 2013, 2013 Asilomar Conference on Signals, Systems and Computers.

[19]  Sonia Vargas-Sierra,et al.  A 151 dB High Dynamic Range CMOS Image Sensor Chip Architecture With Tone Mapping Compression Embedded In-Pixel , 2015, IEEE Sensors Journal.

[20]  Kun Liu,et al.  CMOS low data rate imaging method based on compressed sensing , 2012 .

[21]  O. Yadid-Pecht,et al.  Wide-Dynamic-Range CMOS Image Sensors—Comparative Performance Analysis , 2009, IEEE Transactions on Electron Devices.

[22]  Antoine Dupret,et al.  A 3T or 4T pixel compatible DR extension technique suitable for 3D-IC imagers: A 800×512 and 5μm pixel pitch 2D demonstrator , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[23]  Shree K. Nayar,et al.  Fibonacci Exposure Bracketing for High Dynamic Range Imaging , 2013, 2013 IEEE International Conference on Computer Vision.

[24]  Chih-Cheng Hsieh,et al.  A Linear-Logarithmic CMOS Image Sensor With Pixel-FPN Reduction and Tunable Response Curve , 2014, IEEE Sensors Journal.

[25]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[26]  Pierre Vandergheynst,et al.  Column-separated compressive sampling scheme for low power CMOS image sensors , 2013, 2013 IEEE 11th International New Circuits and Systems Conference (NEWCAS).

[27]  Walter D. Leon-Salas,et al.  An incremental sigma delta converter for compressive sensing applications , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[28]  Thomas Blumensath,et al.  Compressed Sensing With Nonlinear Observations and Related Nonlinear Optimization Problems , 2012, IEEE Transactions on Information Theory.

[29]  Amine Bermak,et al.  Digital pixel sensor with on-line spatial and temporal compression scheme , 2009, Proceedings of the 2009 12th International Symposium on Integrated Circuits.

[30]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[31]  Pierre Vandergheynst,et al.  Universal and efficient compressed sensing by spread spectrum and application to realistic Fourier imaging techniques , 2011, EURASIP J. Adv. Signal Process..

[32]  Pierre Vandergheynst,et al.  On compensating unknown pixel behaviors for image sensors with embedded processing , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[33]  V. Gruev,et al.  Low Power Programmable Current Mode Computational Imaging Sensor , 2012, IEEE Sensors Journal.

[34]  J. Romberg.  Sensing by Random Convolution , 2007, 2007 2nd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing.

[35]  Abbas El Gamal,et al.  CMOS Image Sensor With Per-Column ΣΔ ADC and Programmable Compressed Sensing , 2013, IEEE Journal of Solid-State Circuits.

[36]  Faramarz Farahi,et al.  High dynamic range compressive imaging: a programmable imaging system , 2012 .

[37]  Gunhee Han,et al.  PD-storage dual-capture variable wide dynamic range CMOS image sensor , 2011 .

[38]  Trac D. Tran,et al.  Fast compressive sampling with structurally random matrices , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[39]  Amine Bermak,et al.  Block-based compressive sampling for digital pixel sensor array , 2010, 2nd Asia Symposium on Quality Electronic Design (ASQED).

[40]  Marc Teboulle,et al.  A fast Iterative Shrinkage-Thresholding Algorithm with application to wavelet-based image deblurring , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[41]  David V. Anderson,et al.  Compressive Sensing on a CMOS Separable-Transform Image Sensor , 2010, Proceedings of the IEEE.

[42]  Hyun Seok Hong,et al.  Alternating line high dynamic range imaging , 2011, 2011 17th International Conference on Digital Signal Processing (DSP).

[43]  Laurent Jacques,et al.  A (256×256) pixel 76.7mW CMOS imager/ compressor based on real-time In-pixel compressive sensing , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[44]  Patrick Garda,et al.  Scalable 2D arrays of noise sources for stochastic retina , 1997 .

[45]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..