Per-Pixel Coded Exposure for High-Speed and High-Resolution Imaging Using a Digital Micromirror Device Camera

High-speed photography is an important tool for studying rapid physical phenomena. However, low-frame-rate CCD (charge coupled device) or CMOS (complementary metal oxide semiconductor) camera cannot effectively capture the rapid phenomena with high-speed and high-resolution. In this paper, we incorporate the hardware restrictions of existing image sensors, design the sampling functions, and implement a hardware prototype with a digital micromirror device (DMD) camera in which spatial and temporal information can be flexibly modulated. Combined with the optical model of DMD camera, we theoretically analyze the per-pixel coded exposure and propose a three-element median quicksort method to increase the temporal resolution of the imaging system. Theoretically, this approach can rapidly increase the temporal resolution several, or even hundreds, of times without increasing bandwidth requirements of the camera. We demonstrate the effectiveness of our method via extensive examples and achieve 100 fps (frames per second) gain in temporal resolution by using a 25 fps camera.

[1]  H. Dürr,et al.  Transient ferromagnetic-like state mediating ultrafast reversal of antiferromagnetically coupled spins , 2011, Nature.

[2]  Qiyin Fang,et al.  CMOS Image Sensors for High Speed Applications , 2009, Sensors.

[3]  Motoharu Fujigaki,et al.  Accurate pixel-to-pixel correspondence adjustment in a digital micromirror device camera by using the phase-shifting moiré method. , 2006, Applied optics.

[4]  Edoardo Charbon,et al.  Toward One Giga Frames per Second — Evolution of in Situ Storage Image Sensors , 2013, Sensors.

[5]  Rama Chellappa,et al.  P2C2: Programmable pixel compressive camera for high speed imaging , 2011, CVPR 2011.

[6]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH '06.

[7]  Masatoshi Ishikawa,et al.  Collision avoidance of intelligent vehicle based on networked high-speed vision system , 2014, 2014 11th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[8]  Cuong Vo Le,et al.  Backside illuminated CCD operating at 16,000,000 frames per second with sub-ten-photon sensitivity , 2010 .

[9]  Toshiro Akino,et al.  Toward 100 Mega-Frames per Second: Design of an Ultimate Ultra-High-Speed Image Sensor , 2009, Sensors.

[10]  Albert Stolow,et al.  Time-resolved imaging of purely valence-electron dynamics during a chemical reaction , 2011 .

[11]  Bahram Jalali,et al.  High-throughput single-microparticle imaging flow analyzer , 2012, Proceedings of the National Academy of Sciences.

[12]  Constantin Zopounidis,et al.  Multicriteria classification and sorting methods: A literature review , 2002, Eur. J. Oper. Res..

[13]  Ashok Veeraraghavan,et al.  Flexible Voxels for Motion-Aware Videography , 2010, ECCV.

[14]  Hidenori Akiyama,et al.  Observation of Underwater Streamer Discharges Produced by Pulsed Power Using High-Speed Camera , 2014, IEEE Transactions on Plasma Science.

[15]  Peter Kohl,et al.  Temporal Pixel Multiplexing for simultaneous high-speed high-resolution imaging , 2010, Nature Methods.

[16]  Michal Šorel,et al.  Platform motion blur image restoration system. , 2012, Applied optics.

[17]  Shree K. Nayar,et al.  Efficient Space-Time Sampling with Pixel-Wise Coded Exposure for High-Speed Imaging , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Walter M. Duncan,et al.  Emerging digital micromirror device (DMD) applications , 2003, SPIE MOEMS-MEMS.

[19]  Ramesh Raskar,et al.  Coded Strobing Photography: Compressive Sensing of High Speed Periodic Videos , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Toshiaki Koike-Akino,et al.  Ultra-High-Speed Image Signal Accumulation Sensor , 2010, Sensors.

[21]  Walter F. Kosonocky,et al.  Million-frame-per-second CCD camera system , 1997, Other Conferences.

[22]  Richard M. Osgood,et al.  High-speed, electronically shuttered solid-state imager technology (invited) , 2003 .

[23]  Shree K. Nayar,et al.  Ieee Transactions on Image Processing Computational Cameras: Convergence of Optics and Processing , 2022 .

[24]  Gunnar Ritt,et al.  Automatic Laser Glare Suppression in Electro-Optical Sensors , 2015, Sensors.

[25]  Keigo Hirakawa,et al.  Analysis and processing of pixel binning for color image sensor , 2012, EURASIP J. Adv. Signal Process..

[26]  Albert J. P. Theuwissen,et al.  An image sensor which captures 100 consecutive frames at 1000000 frames/s , 2003 .

[27]  Daehyeok Kim,et al.  A High Speed CMOS Image Sensor with a Novel Digital Correlated Double Sampling and a Differential Difference Amplifier , 2015, Sensors.

[28]  Rihito Kuroda,et al.  A Global-Shutter CMOS Image Sensor With Readout Speed of 1-Tpixel/s Burst and 780-Mpixel/s Continuous , 2013, IEEE Journal of Solid-State Circuits.

[29]  Xing-Hua Qu,et al.  Removal of parasitic image due to metal specularity based on digital micromirror device camera , 2014 .

[30]  S. Ri,et al.  Intensity range extension method for three-dimensional shape measurement in phase-measuring profilometry using a digital micromirror device camera. , 2008, Applied optics.

[31]  Xinqiao Liu,et al.  A 10000 frames/s CMOS digital pixel sensor , 2001, IEEE J. Solid State Circuits.

[32]  Shih-Hao Huang,et al.  Light-Addressed Electrodeposition of Enzyme-Entrapped Chitosan Membranes for Multiplexed Enzyme-Based Bioassays Using a Digital Micromirror Device , 2013, Sensors.