Low bit-rate compression of video and light-field data using coded snapshots and learned dictionaries

The method of coded snapshots has been proposed recently for compressive acquisition of video data to overcome the space-time trade-of inherent in video acquisition. The method involves modulation of the light entering the video camera at different time instants during the exposure period by means of a different and randomly generated code pattern at each of those time instants, followed by integration across time, leading to a single coded snapshot image. Given this image and knowledge of the random codes, it is possible to reconstruct the underlying video frames - by means of sparse coding on a suitably learned dictionary. In this paper, we apply a modified version of this idea, proposed formerly in the compressive sensing literature, to the task of compression of videos and light-field data. At low bit rates, we demonstrate markedly better reconstruction fidelity for the same storage costs, in comparison to JPEG2000 and MPEG-4 (H.264) on light-field and video data respectively. Our technique can cope with overlapping blocks of image data, thereby leading to suppression of block artifacts.

[1]  Marcus A. Magnor,et al.  Data compression for light-field rendering , 2000, IEEE Trans. Circuits Syst. Video Technol..

[2]  Michael Elad,et al.  Compression of facial images using the K-SVD algorithm , 2008, J. Vis. Commun. Image Represent..

[3]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[4]  Moncef Gabbouj,et al.  Sparse/DCT (S/DCT) Two-Layered Representation of Prediction Residuals for Video Coding , 2013, IEEE Transactions on Image Processing.

[5]  Pier Luigi Dragotti,et al.  Distributed compression of the plenoptic function , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[6]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[7]  Xiaoqing Zhu,et al.  Light field compression using disparity-compensated lifting , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[8]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[9]  Takeshi Naemura,et al.  View-Dependent Coding of Light Fields Based on Free-Viewpoint Image Synthesis , 2006, 2006 International Conference on Image Processing.

[10]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[11]  Shree K. Nayar,et al.  Video from a single coded exposure photograph using a learned over-complete dictionary , 2011, 2011 International Conference on Computer Vision.

[12]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[13]  Michael Elad,et al.  Low Bit-Rate Compression of Facial Images , 2007, IEEE Transactions on Image Processing.

[14]  Feng Liu,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries in Wavelet Domain , 2009, 2009 Fifth International Conference on Image and Graphics.

[15]  Bernd Girod,et al.  Light field compression using disparity-compensated lifting , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..