Image Quality Improvements Based on Motion-Based Deblurring for Single-Photon Imaging

Photon counting imaging can be used to capture clearly photon-limited scenes. In photon counting imaging, information on incident photons is obtained as binary frames (bit-plane frames), which are transformed into a multi-bit image in the reconstruction process. In this process, it is necessary to apply a deblurring method to enable the capture of dynamic scenes without motion blur. In this article, a deblurring method for the high-quality bit-plane frame reconstruction of dynamic scenes is proposed. The proposed method involves the deblurring of units of object motion within a scene through the application of motion compensation to pixels sharing the same motions. This method achieves more efficient motion blur suppression than the application of simple deblurring to pixel block or spatial region units. It also applies a novel technique for accurate motion estimation from the bit-plane frame even in photon-limited situations through the statistical evaluation of the temporal variation of photon incidence. In addition to deblurring, our experimental results also revealed that the proposed method can be applied for denoising, which improves the peak signal-to-noise ratio by 1.2 dB. In summary, the proposed method for bit-plane reconstruction achieves high quality imaging even in photon-limited dynamic scenes.

[1]  Edoardo Charbon,et al.  SPAD imagers for super resolution localization microscopy enable analysis of fast fluorophore blinking , 2017, Scientific Reports.

[2]  E. R. Fossum,et al.  What to Do with Sub-Diffraction-Limit (SDL) Pixels?-A Proposal for a Gigapixel Digital Film Sensor (DFS) , 2005 .

[3]  Stanley H. Chan,et al.  Image Reconstruction for Quanta Image Sensors Using Deep Neural Networks , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Stanley H. Chan,et al.  Images from Bits: Non-Iterative Image Reconstruction for Quanta Image Sensors , 2016, Sensors.

[5]  Seungyong Lee,et al.  Video deblurring for hand-held cameras using patch-based synthesis , 2012, ACM Trans. Graph..

[6]  Martin Raab,et al.  "Balls into Bins" - A Simple and Tight Analysis , 1998, RANDOM.

[7]  Ximing Ren,et al.  High-resolution depth profiling using a range-gated CMOS SPAD quanta image sensor. , 2018, Optics express.

[8]  Gries,et al.  The chi-square test for independence , 2022 .

[9]  Vladlen Koltun,et al.  Dynamic Low-light Imaging with Quanta Image Sensors , 2020, ECCV.

[10]  Eric R. Fossum,et al.  The Quanta Image Sensor: Every Photon Counts , 2016, Sensors.

[11]  S. Pellegrini,et al.  A SPAD-Based QVGA Image Sensor for Single-Photon Counting and Quanta Imaging , 2016, IEEE Transactions on Electron Devices.

[12]  Mohit Gupta,et al.  Asynchronous Single-Photon 3D Imaging , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Robert K. Henderson,et al.  Single-Photon Tracking for High-Speed Vision † , 2018, Sensors.

[14]  Martin Vetterli,et al.  Bits From Photons: Oversampled Image Acquisition Using Binary Poisson Statistics , 2011, IEEE Transactions on Image Processing.

[15]  Yue M. Lu,et al.  Learning optimal parameters for binary sensing image reconstruction algorithms , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[16]  G. Sasibhushana Rao,et al.  Chi-Square Based Mobile Radio Propagation Model Analysis and Validation , 2017 .

[17]  Matteo Perenzoni,et al.  A 64 $\times$ 64-Pixels Digital Silicon Photomultiplier Direct TOF Sensor With 100-MPhotons/s/pixel Background Rejection and Imaging/Altimeter Mode With 0.14% Precision Up To 6 km for Spacecraft Navigation and Landing , 2017, IEEE Journal of Solid-State Circuits.

[18]  Andrew J. Holmes,et al.  320×240 oversampled digital single photon counting image sensor , 2014, 2014 Symposium on VLSI Circuits Digest of Technical Papers.

[19]  Rory R. Duncan,et al.  A $256\times256$ , 100-kfps, 61% Fill-Factor SPAD Image Sensor for Time-Resolved Microscopy Applications , 2018, IEEE Transactions on Electron Devices.

[20]  W. Brockherde,et al.  SPAD Smart Pixel for Time-of-Flight and Time-Correlated Single-Photon Counting Measurements , 2012, IEEE Photonics Journal.

[21]  Xiaochun Cao,et al.  Video Deblurring via Semantic Segmentation and Pixel-Wise Non-linear Kernel , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[22]  Robert K. Henderson,et al.  Single Photon Counting Performance and Noise Analysis of CMOS SPAD-Based Image Sensors , 2016, Sensors.

[23]  Stanley H. Chan,et al.  Megapixel Photon-Counting Color Imaging using Quanta Image Sensor , 2019, Optics express.

[24]  Stanley H. Chan,et al.  Efficient image reconstruction for gigapixel quantum image sensors , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[25]  Sabine Süsstrunk,et al.  Image reconstruction in the gigavision camera , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[26]  D. Cumming,et al.  Wireless fluorescence capsule for endoscopy using single photon-based detection , 2015, Scientific Reports.

[27]  Shantanu Gupta,et al.  Quanta burst photography , 2020, ACM Trans. Graph..

[28]  E. Fossum,et al.  Quanta Image Sensor Jot With Sub 0.3e- r.m.s. Read Noise and Photon Counting Capability , 2015, IEEE Electron Device Letters.

[29]  Robert Henderson,et al.  Bit-plane Processing Techniques for Low-Light, High Speed Imaging with a SPAD-based QIS , 2015 .