Indirect Time-of-Flight CMOS Image Sensor With On-Chip Background Light Cancelling and Pseudo-Four-Tap/Two-Tap Hybrid Imaging for Motion Artifact Suppression

This article presents a 320 $\times $ 240 indirect time-of-flight (iToF) CMOS image sensor (CIS) with on-chip motion artifact suppression and background light cancelling (BGLC). The proposed iToF CIS uses a backside-illuminated trident pinned photodiode (PPD) that assists charge transfer with a built-in lateral electric field for enhanced depth accuracy. To overcome the limitation of the conventional iToF CIS that exhibits motion artifact, we propose a pseudo-four-tap (P4-tap) demodulation method with alternate phase driving using a conventional two-tap pixel structure with a high fill factor of over 43%. In addition, by combining the advantages of both the P4-tap and conventional two-tap demodulation schemes, we propose hybrid depth imaging with reduced motion artifact for moving objects, while providing high-depth precision for the static background. For outdoor mobile applications of the iToF CIS, we integrated on-chip BGLC circuits to eliminate the BGL-induced depth error. A prototype chip is fabricated using a 90-nm backside illumination (BSI) CIS process. The BSI trident PPD enabled a low depth error of under 0.54% over the range of 0.75–4 m, with a modulation frequency of 100 MHz. Motion artifact was suppressed at 60 frames/s of hybrid depth imaging owing to the proposed P4-tap scheme. With the on-chip BGLC circuit, the experimental results demonstrate a 0.55% depth error at a 1-m distance, even under 120-klx BGL illumination.

[1]  P.-A. Besse,et al.  Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes , 2005, IEEE Journal of Solid-State Circuits.

[2]  Shoji Kawahito,et al.  CMOS Lock-in Pixel Image Sensors with Lateral Eelectric Field Cotrol for Time-Resolved Imaging , 2013 .

[3]  Chang-Yeong Kim,et al.  A 1920×1080 3.65μm-pixel 2D/3D image sensor with split and binning pixel structure in 0.11pm standard CMOS , 2012, 2012 IEEE International Solid-State Circuits Conference.

[4]  S. Aoyama,et al.  A Time-of-Flight Range Image Sensor With Background Canceling Lock-in Pixels Based on Lateral Electric Field Charge Modulation , 2015, IEEE Journal of the Electron Devices Society.

[5]  Abbas El Gamal,et al.  Analysis of temporal noise in CMOS photodiode active pixel sensor , 2001, IEEE J. Solid State Circuits.

[6]  Assim Boukhayma,et al.  Ultra Low Noise CMOS Image Sensors , 2017 .

[7]  Larry Li,et al.  Time-of-Flight Camera - An Introduction , 2014 .

[8]  Pierre Magnan,et al.  High Speed Dual Port Pinned-photodiode for Time-Of-Flight Imaging , 2009 .

[9]  Jihoon Park,et al.  5.4 A Dynamic Pseudo 4-Tap CMOS Time-of-Flight Image Sensor with Motion Artifact Suppression and Background Light Cancelling Over 120klux , 2020, 2020 IEEE International Solid- State Circuits Conference - (ISSCC).

[10]  Hyunsurk Ryu,et al.  A VGA Indirect Time-of-Flight CMOS Image Sensor With 4-Tap 7- $\mu$ m Global-Shutter Pixel and Fixed-Pattern Phase Noise Self-Compensation , 2020, IEEE Journal of Solid-State Circuits.

[11]  Edoardo Charbon,et al.  A Modular, Direct Time-of-Flight Depth Sensor in 45/65-nm 3-D-Stacked CMOS Technology , 2019, IEEE Journal of Solid-State Circuits.

[12]  Martin Wolf,et al.  A 30-frames/s, $252\times144$ SPAD Flash LiDAR With 1728 Dual-Clock 48.8-ps TDCs, and Pixel-Wise Integrated Histogramming , 2019, IEEE Journal of Solid-State Circuits.

[13]  Edoardo Charbon,et al.  A 128×128 Single-Photon Imager with on-Chip Column-Level 10b Time-to-Digital Converter Array Capable of 97ps Resolution , 2008, 2008 IEEE International Solid-State Circuits Conference - Digest of Technical Papers.

[14]  E. Charbon,et al.  A Single Photon Avalanche Diode Implemented in 130-nm CMOS Technology , 2007, IEEE Journal of Selected Topics in Quantum Electronics.

[15]  S. Kawahito,et al.  Noise analysis of high-gain, low-noise column readout circuits for CMOS image sensors , 2004, IEEE Transactions on Electron Devices.

[16]  Edoardo Charbon,et al.  A 32×32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging , 2009, 2009 IEEE Custom Integrated Circuits Conference.

[17]  Tzu-Hsiang Hsu,et al.  A CMOS Time-of-Flight Depth Image Sensor With In-Pixel Background Light Cancellation and Phase Shifting Readout Technique , 2018, IEEE Journal of Solid-State Circuits.

[18]  Jaehyuk Choi,et al.  A 3-D Camera With Adaptable Background Light Suppression Using Pixel-Binning and Super-Resolution , 2014, IEEE Journal of Solid-State Circuits.

[19]  Takeshi Yamazaki,et al.  320×240 Back-illuminated 10μm CAPD pixels for high speed modulation Time-of-Flight CMOS image sensor , 2017, 2017 Symposium on VLSI Circuits.

[20]  David Stoppa,et al.  Optimization of Pinned Photodiode Pixels for High-Speed Time of Flight Applications , 2018, IEEE Journal of the Electron Devices Society.

[21]  Lale Akarun,et al.  Real time hand pose estimation using depth sensors , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[22]  Cristiano Niclass,et al.  A 100m-range 10-frame/s 340×96-pixel time-of-flight depth sensor in 0.18μm CMOS , 2011, 2011 Proceedings of the ESSCIRC (ESSCIRC).

[23]  Matteo Perenzoni,et al.  A 32×32-pixel time-resolved single-photon image sensor with 44.64μm pitch and 19.48% fill-factor with on-chip row/frame skipping features reaching 800kHz observation rate for quantum physics applications , 2018, 2018 IEEE International Solid - State Circuits Conference - (ISSCC).

[24]  Youngcheol Chae,et al.  A 64×64 APD-Based ToF Image Sensor with Background Light Suppression up to 200 klx Using In-Pixel Auto-Zeroing and Chopping , 2019, 2019 Symposium on VLSI Circuits.

[25]  Hyunsurk Ryu,et al.  A 640×480 Indirect Time-of-Flight CMOS Image Sensor with 4-tap 7-μm Global-Shutter Pixel and Fixed-Pattern Phase Noise Self-Compensation Scheme , 2019, 2019 Symposium on VLSI Circuits.

[26]  Edward Fisher,et al.  A Reconfigurable Single-Photon-Counting Integrating Receiver for Optical Communications , 2013, IEEE Journal of Solid-State Circuits.

[27]  Iskender Agi,et al.  IMpixel 65nm BSI 320MHz demodulated TOF Image sensor with 3μm global shutter pixels and analog binning , 2018, 2018 IEEE International Solid - State Circuits Conference - (ISSCC).

[28]  Seong-Jin Kim,et al.  A CMOS Image Sensor Based on Unified Pixel Architecture With Time-Division Multiplexing Scheme for Color and Depth Image Acquisition , 2012, IEEE Journal of Solid-State Circuits.

[29]  Andrew J. Holmes,et al.  11.5 A time-correlated single-photon-counting sensor with 14GS/S histogramming time-to-digital converter , 2015, 2015 IEEE International Solid-State Circuits Conference - (ISSCC) Digest of Technical Papers.

[30]  Andrew D. Payne,et al.  A 0.13 μm CMOS System-on-Chip for a 512 × 424 Time-of-Flight Image Sensor With Multi-Frequency Photo-Demodulation up to 130 MHz and 2 GS/s ADC , 2015, IEEE Journal of Solid-State Circuits.

[31]  Shoji Kawahito,et al.  A 10 ps Time-Resolution CMOS Image Sensor With Two-Tap True-CDS Lock-In Pixels for Fluorescence Lifetime Imaging , 2016, IEEE Journal of Solid-State Circuits.

[32]  Satoshi Kondo,et al.  A 20-ch TDC/ADC Hybrid Architecture LiDAR SoC for 240 $\times$ 96 Pixel 200-m Range Imaging With Smart Accumulation Technique and Residue Quantizing SAR ADC , 2018, IEEE Journal of Solid-State Circuits.

[33]  Jaehyuk Choi,et al.  Design of a Time-of-Flight Sensor With Standard Pinned-Photodiode Devices Toward 100-MHz Modulation Frequency , 2019, IEEE Access.