Cmos Image Sensors Dynamic Range and Snr Enhancement via Statistical Signal Processing

ii I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Abbas El Gamal (Principal Adviser) I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Abstract Most of today's video and digital cameras use CCD image sensors, where the electric charge collected by the photodetector array during exposure time is serially shifted out of the sensor chip resulting in slow readout speed and high power consumption. Recently developed CMOS image sensors, by comparison, are read out non-destructively and in a manner similar to a digital memory and can thus be operated at very high frame rates. A CMOS image sensor can also be integrated with other camera functions on the same chip ultimately leading to a single-chip digital camera with very compact size, low power consumption and additional functionality. CMOS image sensors , however, generally suffer from lower dynamic range than CCDs due to their high read noise and non-uniformity. Moreover, as sensor design follows CMOS technology scaling, well capacity will continue to decrease, eventually resulting in unacceptably low SNR. In this dissertation, new pixel architectures and algorithms are presented that enhance the dynamic range and SNR of CMOS image sensors by utilizing their high speed readout and integration advantages. The thesis is divided into three parts. First, a 352 × 288 Digital Pixel Sensor (DPS) chip with per-pixel single-slope ADC and dynamic memory fabricated in a standard digital 0.18µm CMOS process is presented that demonstrates the high speed potential and scaling advantage of CMOS image sensors. The chip performs " snapshot " image acquisition at continuous rate of 10, 000 frames/s or 1 Gpixels/s. Second, an algorithm based on statistical signal processing techniques is presented that synthesizes a high dynamic range, motion blur free, still image or video sequence from multiple image captures. The algorithm iv is recursive and consists of two main procedures – photocurrent estimation and mo-tion/saturation detection. Photocurrent estimation is used to reduce read noise and thus to enhance dynamic range at …

[1]  Hao Min,et al.  Modeling and estimation of FPN components in CMOS image sensors , 1998, Electronic Imaging.

[2]  T. Kamata,et al.  A 1280 × 980 pixel CCD image sensor , 1986, 1986 IEEE International Solid-State Circuits Conference. Digest of Technical Papers.

[3]  N. Kokusenya,et al.  A 1 mm 50 K-pixel IT CCD image sensor for miniature camera system , 1998, 1998 IEEE International Solid-State Circuits Conference. Digest of Technical Papers, ISSCC. First Edition (Cat. No.98CH36156).

[4]  David Renshaw,et al.  CMOS image sensors for multimedia applications , 1993, Proceedings of IEEE Custom Integrated Circuits Conference - CICC '93.

[5]  Theodore I. Kamins,et al.  Device Electronics for Integrated Circuits , 1977 .

[6]  Cyril Cavadore,et al.  Recent experimental results from a CMOS active pixel image sensor with photodiode and photogate pixels , 1996, Advanced Imaging and Network Technologies.

[7]  James R. Janesick,et al.  CCD transfer method: standard for absolute performance of CCDs and digital CCD camera systems , 1997, Electronic Imaging.

[8]  Eric R. Fossum,et al.  Active pixel sensors: are CCDs dinosaurs? , 1993, Electronic Imaging.

[9]  Bedabrata Pain,et al.  Progress in CMOS active pixel image sensors , 1994, Electronic Imaging.

[10]  B.J. Hosticka,et al.  A high dynamic range CMOS image sensor for automotive applications , 2000, Proceedings of the 25th European Solid-State Circuits Conference.

[11]  Hsin-Fu Tseng,et al.  A High Performance 1200 X 400 CCD Array For Astronomy Applications , 1989, Photonics West - Lasers and Applications in Science and Engineering.

[12]  J. Schemmel,et al.  A self-calibrating single-chip CMOS camera with logarithmic response , 2001, IEEE J. Solid State Circuits.

[13]  Abbas El Gamal,et al.  Techniques for pixel-level analog-to-digital conversion , 1998, Defense, Security, and Sensing.

[14]  E.R. Fossum,et al.  A 1.2 V micropower CMOS active pixel image sensor for portable applications , 2000, 2000 IEEE International Solid-State Circuits Conference. Digest of Technical Papers (Cat. No.00CH37056).

[15]  B. Hosticka,et al.  A locally adaptive CMOS image sensor with 90 dB dynamic range , 1999, 1999 IEEE International Solid-State Circuits Conference. Digest of Technical Papers. ISSCC. First Edition (Cat. No.99CH36278).

[16]  Kiyoharu Aizawa,et al.  A computational image sensor with adaptive pixel-based integration time , 2001 .

[17]  Eugenio Culurciello,et al.  High dynamic range, arbitrated address event representation digital imager , 2001, ISCAS 2001. The 2001 IEEE International Symposium on Circuits and Systems (Cat. No.01CH37196).

[18]  Sandor L. Barna,et al.  A High Dynamic Range CMOS APS Image Sensor , 2008 .

[19]  Abbas El Gamal,et al.  Method for estimating quantum efficiency for CMOS image sensors , 1998, Electronic Imaging.

[20]  Misha Mahowald,et al.  A silicon model of early visual processing , 1993, Neural Networks.

[21]  T. Yamada,et al.  A 1/2 inch 1.3 M-pixel progressive-scan IT-CCD for still and motion picture applications , 1998, 1998 IEEE International Solid-State Circuits Conference. Digest of Technical Papers, ISSCC. First Edition (Cat. No.98CH36156).

[22]  Abbas El Gamal,et al.  A Nyquist rate pixel level ADC for CMOS image sensors , 1998, Proceedings of the IEEE 1998 Custom Integrated Circuits Conference (Cat. No.98CH36143).

[23]  Eric R. Fossum,et al.  CMOS image sensors: electronic camera on a chip , 1995, Proceedings of International Electron Devices Meeting.

[24]  Orly Yadid-Pecht,et al.  Wide intrascene dynamic range CMOS APS using dual sampling , 1997 .

[25]  Bruce A. Wooley,et al.  A 128×128-pixel standard-CMOS image sensor with electronic shutter , 1996 .

[26]  Xinqiao Liu,et al.  Simultaneous image formation and motion blur restoration via multiple capture , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[27]  Bryan D. Ackland,et al.  A 200-mW, 3.3-V, CMOS Color Camera IC Producing 352 288 24-b Video at 30 Frames/s , 1998 .

[28]  Mohammed Ismail,et al.  Analog VLSI Implementation of Neural Systems , 2011, The Kluwer International Series in Engineering and Computer Science.

[29]  B. Dierickx,et al.  Pixel structure with logarithmic response for intelligent and flexible imager architectures , 1992, ESSDERC '92: 22nd European Solid State Device Research conference.

[30]  Barry Fowler,et al.  A 640×512 CMOS Image Sensor with Ultra Wide Dynamic Range Floating-Point Pixel-Level ADC , 1999 .

[31]  B. Mansoorian,et al.  A high speed, 500 frames/s, 1024/spl times/1024 CMOS active pixel sensor , 1999, 1999 Symposium on VLSI Circuits. Digest of Papers (IEEE Cat. No.99CH36326).

[32]  R. Latham,et al.  High-speed imaging of the pulsed-field flashover of an alumina ceramic in vacuum , 1995 .

[33]  Carver A. Mead,et al.  Analog VLSI Phototransduction by continuous-time, adaptive, logarithmic photoreceptor circuits , 1995 .

[34]  J. Bosiers,et al.  An S-VHS compatible 1/3" color FT-CCD imager with low dark current by surface pinning , 1995 .

[35]  Woodward Yang,et al.  A wide-dynamic-range, low-power photosensor array , 1994, Proceedings of IEEE International Solid-State Circuits Conference - ISSCC '94.

[36]  W. D. Washkurak,et al.  Current-mediated, current-reset 768/spl times/512 active pixel sensor array , 1997, 1997 IEEE International Solids-State Circuits Conference. Digest of Technical Papers.

[37]  Stuart Anderson,et al.  On-Chip CMOS Sensors for VLSI Imaging Systems , 1991, Conference on Advanced Research in VLSI.

[38]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[39]  N. Tanaka,et al.  A novel bipolar imaging device with self-noise-reduction capability , 1989 .

[40]  S. Felici,et al.  Response properties of a foveated space-variant CMOS image sensor , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[41]  Robert G. Meyer,et al.  Analysis and Design of Analog Integrated Circuits , 1993 .

[42]  E.R. Fossum,et al.  128 Mb/s multiport CMOS binary active-pixel image sensor , 1996, 1996 IEEE International Solid-State Circuits Conference. Digest of TEchnical Papers, ISSCC.

[43]  Abbas El Gamal,et al.  Comparative analysis of SNR for image sensors with enhanced dynamic range , 1999, Electronic Imaging.

[44]  Orly Yadid-Pecht,et al.  Wide-dynamic-range sensors , 1999 .

[45]  Xinqiao Liu,et al.  Pixel Sensor with Pixel-Level Memory , 2001 .

[46]  S. Decker,et al.  A 256/spl times/256 CMOS imaging array with wide dynamic range pixels and column-parallel digital output , 1998, 1998 IEEE International Solid-State Circuits Conference. Digest of Technical Papers, ISSCC. First Edition (Cat. No.98CH36156).

[47]  Takeo Kanade,et al.  A sorting image sensor: an example of massively parallel intensity-to-time processing for low-latency computational sensors , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[48]  Abbas El Gamal,et al.  Pixel-level processing: why, what, and how? , 1999, Electronic Imaging.

[49]  S. Wuu,et al.  A high performance active pixel sensor with 0.18um CMOS color imager technology , 2001, International Electron Devices Meeting. Technical Digest (Cat. No.01CH37224).

[50]  Harold W. Sorenson,et al.  Parameter estimation: Principles and problems , 1980 .

[51]  A. Theuwissen,et al.  Solid-State Imaging with Charge-Coupled Devices , 1995 .

[52]  Sarit Chen,et al.  Adaptive sensitivity CCD image sensor , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 2 - Conference B: Computer Vision & Image Processing. (Cat. No.94CH3440-5).

[53]  Kiyoharu Aizawa,et al.  Motion adaptive image sensor , 1998, Proceedings of 1998 Asia and South Pacific Design Automation Conference.

[54]  Bart Dierickx,et al.  Random addressable active pixel image sensors , 1996, Advanced Imaging and Network Technologies.

[55]  Takahisa Ueno,et al.  A CMOS image sensor with a simple FPN-reduction technology and a hole accumulated diode , 2000, 2000 IEEE International Solid-State Circuits Conference. Digest of Technical Papers (Cat. No.00CH37056).

[56]  Fossum,et al.  256x256 CMOS ACTIVE PIXEL SENSOR CAMERA-ON-A-CHIP , 1997 .

[57]  Abbas El Gamal,et al.  Characterization of CMOS image sensors with Nyquist rate pixel-level ADC , 1999, Electronic Imaging.

[58]  D. Renshaw,et al.  A single chip video camera with on-chip automatic exposure control , 1991 .

[59]  Takeo Kanade,et al.  Temporal photoreception for adaptive dynamic range image sensing and encoding , 1998, Neural Networks.

[60]  A.S. Sedra,et al.  Analog MOS integrated circuits for signal processing , 1987, Proceedings of the IEEE.

[61]  Narendra Ahuja,et al.  High dynamic range panoramic imaging , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[62]  Donald James Baxter,et al.  800-thousand-pixel color CMOS sensor for consumer still cameras , 1997, Electronic Imaging.

[63]  J. Miteran,et al.  A low cost and intelligent video surveillance system , 1995, 1995 Proceedings of the IEEE International Symposium on Industrial Electronics.

[64]  Shree K. Nayar,et al.  High dynamic range imaging: spatially varying pixel exposures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[65]  M. Fardo,et al.  High speed imaging with a tapped solid state sensor , 1989 .

[66]  Hon-Sum Philip Wong,et al.  Technology and device scaling considerations for CMOS imagers , 1996 .

[67]  B.J. Hosticka,et al.  A CMOS image sensor for high-speed imaging , 2000, 2000 IEEE International Solid-State Circuits Conference. Digest of Technical Papers (Cat. No.00CH37056).

[68]  E. Okamoto,et al.  A progressive scan CCD imager for DSC applications , 2000, 2000 IEEE International Solid-State Circuits Conference. Digest of Technical Papers (Cat. No.00CH37056).

[69]  Chris J. Schaeffer,et al.  Memory read-out approach for a 0.5-μm CMOS image sensor , 1998, Electronic Imaging.

[70]  Bedabrata Pain,et al.  Optimization of noise and responsivity in CMOS active pixel sensors for detection of ultralow-light levels , 1997, Electronic Imaging.

[71]  Carver A. Mead.  A Sensitive Electronic Photoreceptor , 1985 .

[72]  M. Yamawaki,et al.  A pixel size shrinkage of amplified MOS imager with two-line mixing , 1996 .

[73]  Robert G. Root,et al.  Repetitively pulsed plasma illumination source improvements , 1999, Electronic Imaging.

[74]  Thomas F. Knight,et al.  Design of an integrated optical sensor with on-chip preprocessing , 1983 .

[75]  Abbas El Gamal,et al.  Active pixel sensors fabricated in a standard 0.18-μm CMOS technology , 2001, IS&T/SPIE Electronic Imaging.

[76]  Kiyoharu Aizawa,et al.  New design and implementation of adaptive-integration-time image sensor , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[77]  G. Michon,et al.  Charge injection imaging , 1973 .

[78]  Ali H. Sayed,et al.  Linear Estimation (Information and System Sciences Series) , 2000 .

[79]  Tadahiro Ohmi,et al.  A 310 k pixel bipolar imager (BASIS) , 1989, IEEE International Solid-State Circuits Conference, 1989 ISSCC. Digest of Technical Papers.

[80]  Eric Belhaire,et al.  High-current large-bandwidth photosensor on standard CMOS processes , 1996, Advanced Imaging and Network Technologies.

[81]  Brian A. Wandell,et al.  How small should pixel size be? , 2000, Electronic Imaging.

[82]  S. Sze.  Semiconductor Devices: Physics and Technology , 1985 .

[83]  B. Fowler,et al.  A CMOS area image sensor with pixel-level A/D conversion , 1994, Proceedings of IEEE International Solid-State Circuits Conference - ISSCC '94.

[84]  S. Kavadias,et al.  A logarithmic response CMOS image sensor with on-chip calibration , 2000, IEEE Journal of Solid-State Circuits.

[85]  Hao Min,et al.  Test structures for characterization and comparative analysis of CMOS image sensors , 1996, Advanced Imaging and Network Technologies.

[86]  Abbas El Gamal,et al.  Integration of image capture and processing: beyond single-chip digital camera , 2001, IS&T/SPIE Electronic Imaging.

[87]  Eric R. Fossum,et al.  Ultra-low-power imaging systems using CMOS image sensor technology , 1994, Optics & Photonics.

[88]  Abbas El Gamal,et al.  Photocurrent estimation from multiple nondestructive samples in CMOS image sensor , 2001, IS&T/SPIE Electronic Imaging.

[89]  Orly Yadid-Pecht,et al.  CMOS APS with autoscaling and customized wide dynamic range , 1999 .