Spatiotemporal compressed sensing for video compression

We present a hardware-friendly spatiotemporal compressed sensing framework for video compression. The spatiotemporal compressed sensing incorporates random sampling in both spatial and temporal domain to encode the video scene into a single coded image. During decoding, the video is reconstructed using dictionary learning and sparse recovery. The evaluation results demonstrate the proposed approach can achieve high compression rate (10 : 1–30 : 1) and robustness reconstruction quality (> 20dB) on noisy database. Additionally, it also enables power efficient and real-time CMOS implementation (0.7 nJ/pixel).

[1]  Lien-Fei Chen,et al.  A 0.5 nJ/Pixel 4 K H.265/HEVC Codec LSI for Multi-Format Smartphone Applications , 2016, IEEE Journal of Solid-State Circuits.

[2]  Peter D. Symes.  Video Compression , 1998 .

[3]  Guido Schuster,et al.  High spatio-temporal resolution video with compressed sensing. , 2015, Optics express.

[4]  Guillermo Sapiro,et al.  Temporal Compressive Sensing for Video , 2015 .

[5]  Alcino J. Silva,et al.  A shared neural ensemble links distinct contextual memories encoded close in time , 2016, Nature.

[6]  Kartikeya Murari,et al.  A Miniaturized Platform for Laser Speckle Contrast Imaging , 2012, IEEE Transactions on Biomedical Circuits and Systems.

[7]  Lawrence Carin,et al.  Spectral-temporal compressive imaging. , 2015, Optics letters.

[8]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[9]  Jie Zhang,et al.  Compact all-CMOS spatiotemporal compressive sensing video camera with pixel-wise coded exposure. , 2016, Optics express.

[10]  Guillermo Sapiro,et al.  Coded aperture compressive temporal imaging , 2013, Optics express.

[11]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[12]  Shree K. Nayar,et al.  Video from a single coded exposure photograph using a learned over-complete dictionary , 2011, 2011 International Conference on Computer Vision.

[13]  Liang-Gee Chen,et al.  Video Compression , 2010, Handbook of Signal Processing Systems.

[14]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[15]  Shree K. Nayar,et al.  Efficient Space-Time Sampling with Pixel-Wise Coded Exposure for High-Speed Imaging , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[17]  S. Mochizuki,et al.  A 64 mW High Picture Quality H.264/MPEG-4 Video Codec IP for HD Mobile Applications in 90 nm CMOS , 2008, IEEE Journal of Solid-State Circuits.