Random coded sampling for high-speed HDR video

We propose a novel method for capturing high-speed, high dynamic range video with a single low-speed camera using a coded sampling technique. Traditional video cameras use a constant full-frame exposure time, which makes temporal super-resolution difficult due to the ill-posed nature of inverting the sampling operation. Our method samples at the same rate as the traditional low-speed camera but uses random per-pixel exposure times and offsets. By exploiting temporal and spatial redundancy in the video, we can reconstruct a high-speed video from the coded input. Furthermore, the different exposure times used in our sampling scheme enable us to obtain a higher dynamic range than a traditional camera or other temporal superresolution methods. We validate our approach using simulation and provide a detailed discussion on how to make a hardware implementation. In particular, we believe that our approach can maintain a 100% light throughput similarly to existing cameras and can be implemented on a single chip, making it suitable for small form factors.

[1]  Ashok Veeraraghavan,et al.  Optimal coded sampling for temporal super-resolution , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Shree K. Nayar,et al.  Video from a single coded exposure photograph using a learned over-complete dictionary , 2011, 2011 International Conference on Computer Vision.

[3]  Tomoo Mitsunaga,et al.  Coded rolling shutter photography: Flexible space-time sampling , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[4]  Truong Nguyen,et al.  HDR Imaging From Differently Exposed Multiview Videos , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[5]  Pradeep Sen,et al.  A versatile HDR video production system , 2011, ACM Trans. Graph..

[6]  Aswin C. Sankaranarayanan,et al.  CS-MUVI: Video compressive sensing for spatial-multiplexing cameras , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[7]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[8]  Rama Chellappa,et al.  P2C2: Programmable pixel compressive camera for high speed imaging , 2011, CVPR 2011.

[9]  L. McMillan,et al.  Video enhancement using per-pixel virtual exposures , 2005, SIGGRAPH 2005.

[10]  Li Zhang,et al.  Denoising vs. deblurring: HDR imaging techniques using moving cameras , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Leonard McMillan,et al.  Video enhancement using per-pixel virtual exposures , 2005, ACM Trans. Graph..

[12]  T. Poonnen,et al.  A CMOS video sensor for High Dynamic Range (HDR) imaging , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[13]  Aswin C. Sankaranarayanan,et al.  Flutter Shutter Video Camera for compressive sensing of videos , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[14]  Ashok Veeraraghavan,et al.  Flexible Voxels for Motion-Aware Videography , 2010, ECCV.

[15]  Shree K. Nayar,et al.  Enhancing resolution along multiple imaging dimensions using assorted pixels , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Michal Irani,et al.  Space-time super-resolution from a single video , 2011, CVPR 2011.

[17]  Michael B. Wakin,et al.  A multiscale framework for Compressive Sensing of video , 2009, 2009 Picture Coding Symposium.

[18]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..