Video compressive sensing with on-chip programmable subsampling

The maximum achievable frame-rate for a video camera is limited by the sensor's pixel readout rate. The same sensor may achieve either a slow frame-rate at full resolution (e.g., 60 fps at 4 Mpixel resolution) or a fast frame-rate at low resolution (e.g., 240 fps at 1 Mpixel resolution). Higher frame-rates are achieved using pixel readout modes (e.g., subsampling or binning) that sacrifice spatial for temporal resolution within a fixed bandwidth. A number of compressive video cameras have been introduced to overcome this fixed bandwidth constraint and achieve high frame-rates without sacrificing spatial resolution. These methods use electro-optic components (e.g., LCoS, DLPs, piezo actuators) to introduce high speed spatio-temporal multiplexing in captured images. Full resolution, high speed video is then restored by solving an undetermined system of equations using a sparse regularization framework. In this work, we introduce the first all-digital temporal compressive video camera that uses custom subsampling modes to achieve spatio-temporal multiplexing. Unlike previous compressive video cameras, ours requires no additional optical components, enabling it to be implemented in a compact package such as a mobile camera module. We demonstrate results using a TrueSense development kit with a 12 Mpixel sensor and programmable FPGA read out circuitry.

[1]  Shree K. Nayar,et al.  Efficient Space-Time Sampling with Pixel-Wise Coded Exposure for High-Speed Imaging , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Aswin C. Sankaranarayanan,et al.  CS-MUVI: Video compressive sensing for spatial-multiplexing cameras , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[3]  Ashok Veeraraghavan,et al.  Flexible Voxels for Motion-Aware Videography , 2010, ECCV.

[4]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[5]  Guillermo Sapiro,et al.  Coded aperture compressive temporal imaging , 2013, Optics express.

[6]  Rama Chellappa,et al.  P2C2: Programmable pixel compressive camera for high speed imaging , 2011, CVPR 2011.

[7]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[8]  Ashok Veeraraghavan,et al.  Optimal coded sampling for temporal super-resolution , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Chiye Li,et al.  Single-shot compressed ultrafast photography at one hundred billion frames per second , 2014, Nature.

[10]  Aswin C. Sankaranarayanan,et al.  Flutter Shutter Video Camera for compressive sensing of videos , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[11]  Shree K. Nayar,et al.  Motion-based motion deblurring , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Gil Bub,et al.  Pixel Multiplexing for Simultaneous High Resolution High Speed Image Capture , 2010 .