Block-Based Compressed Sensing of Images and Video

A number of techniques for the compressed sensing of imagery are surveyed. Various imaging media are considered, including still images, motion video, as well as multiview image sets and multiview video. A particular emphasis is placed on block-based compressed sensing due to its advantages in terms of both lightweight reconstruction complexity as well as a reduced memory burden for the random-projection measurement operator. For multiple-image scenarios, including video and multiview imagery, motion and disparity compensation is employed to exploit frame-to-frame redundancies due to object motion and parallax, resulting in residual frames which are more compressible and thus more easily reconstructed from compressed-sensing measurements. Extensive experimental comparisons evaluate various prominent reconstruction algorithms for still-image, motion-video, and multiview scenarios in terms of both reconstruction quality as well as computational complexity.

[1]  K. Ramchandran,et al.  Distributed video coding in wireless sensor networks , 2006, IEEE Signal Processing Magazine.

[2]  James E. Fowler,et al.  Video Coding Using a Complex Wavelet Transform and Set Partitioning , 2007, IEEE Signal Processing Letters.

[3]  Rodney G. Vaughan,et al.  The theory of bandpass sampling , 1991, IEEE Trans. Signal Process..

[4]  J. Fowler,et al.  Wavelets in Source Coding , Communications , and Networks : An Overview , 2007 .

[5]  Ephraim Feig,et al.  Fast algorithms for the discrete cosine transform , 1992, IEEE Trans. Signal Process..

[6]  Yaakov Tsaig,et al.  Extensions of compressed sensing , 2006, Signal Process..

[7]  Xiangjun Zhang,et al.  Model-Guided Adaptive Recovery of Compressive Sensing , 2009, 2009 Data Compression Conference.

[8]  James E. Fowler,et al.  An Overview on Wavelets in Source Coding, Communications, and Networks , 2007, EURASIP J. Image Video Process..

[9]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[10]  Richard G. Baraniuk,et al.  Compressive imaging for video representation and coding , 2006 .

[11]  Levent Sendur,et al.  Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency , 2002, IEEE Trans. Signal Process..

[12]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[13]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[14]  Volkan Cevher,et al.  Low-Dimensional Models for Dimensionality Reduction and Signal Recovery: A Geometric Perspective , 2010, Proceedings of the IEEE.

[15]  Aggelos K. Katsaggelos,et al.  Bayesian Compressive Sensing Using Laplace Priors , 2010, IEEE Transactions on Image Processing.

[16]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[17]  Stephen J. Wright,et al.  Computational Methods for Sparse Solution of Linear Inverse Problems , 2010, Proceedings of the IEEE.

[18]  John W. Woods,et al.  Motion-compensated 3-D subband coding of video , 1999, IEEE Trans. Image Process..

[19]  Paul Hasler,et al.  Compressive Sensing on a CMOS Separable-Transform Image Sensor , 2010, Proc. IEEE.

[20]  Feng Wu,et al.  Image representation by compressive sensing for visual sensor networks , 2010, J. Vis. Commun. Image Represent..

[21]  Hans-Dieter Lüke,et al.  The origins of the sampling theorem , 1999, IEEE Commun. Mag..

[22]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[23]  Trac D. Tran,et al.  Distributed Compressed Video Sensing , 2009, 2009 43rd Annual Conference on Information Sciences and Systems.

[24]  James E. Fowler,et al.  Multiscale block compressed sensing with smoothed projected Landweber reconstruction , 2011, 2011 19th European Signal Processing Conference.

[25]  Trac D. Tran,et al.  Fast compressive imaging using scrambled block Hadamard ensemble , 2008, 2008 16th European Signal Processing Conference.

[26]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[27]  Jong Chul Ye,et al.  k‐t FOCUSS: A general compressed sensing framework for high resolution dynamic MRI , 2009, Magnetic resonance in medicine.

[28]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[29]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[30]  Xu Chen,et al.  Joint reconstruction of compressed multi-view images , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[31]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[32]  Thong T. Do,et al.  Sparsity adaptive matching pursuit algorithm for practical compressed sensing , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[33]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[34]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[35]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.

[36]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[37]  Bernd Girod,et al.  Motion-compensating prediction with fractional-pel accuracy , 1993, IEEE Trans. Commun..

[38]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.

[39]  Michael B. Wakin,et al.  A multiscale framework for Compressive Sensing of video , 2009, 2009 Picture Coding Symposium.

[40]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[41]  Ali Bilgin,et al.  Compressed sensing using a Gaussian Scale Mixtures model in wavelet domain , 2010, 2010 IEEE International Conference on Image Processing.

[42]  Namrata Vaswani,et al.  Kalman filtered Compressed Sensing , 2008, 2008 15th IEEE International Conference on Image Processing.

[43]  Yonina C. Eldar,et al.  Structured Compressed Sensing: From Theory to Applications , 2011, IEEE Transactions on Signal Processing.

[44]  Thomas Maugey,et al.  Disparity-compensated compressed-sensing reconstruction for multiview images , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[45]  Béatrice Pesquet-Popescu,et al.  Three-dimensional lifting schemes for motion compensated video compression , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[46]  Thomas Maugey,et al.  Compressed sensing of multiview images using disparity compensation , 2010, 2010 IEEE International Conference on Image Processing.

[47]  Peter Boesiger,et al.  Compressed sensing in dynamic MRI , 2008, Magnetic resonance in medicine.

[48]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[49]  Ali Bilgin,et al.  Motion-compensated compressed sensing for dynamic imaging , 2010, Optical Engineering + Applications.

[50]  N. Kingsbury.  Complex Wavelets for Shift Invariant Analysis and Filtering of Signals , 2001 .

[51]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[52]  H. D. Luke,et al.  The origins of the sampling theorem , 1999 .

[53]  M. Ohta,et al.  An overlapped block motion compensation for high quality motion picture coding , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[54]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[55]  Qin Hao,et al.  A dictionary generation scheme for block-based compressed video sensing , 2011, 2011 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC).

[56]  Namrata Vaswani,et al.  LS-CS-Residual (LS-CS): Compressive Sensing on Least Squares Residual , 2009, IEEE Transactions on Signal Processing.

[57]  Thomas S. Huang,et al.  Distributed Video Coding using Compressive Sampling , 2009, 2009 Picture Coding Symposium.

[58]  James E. Fowler,et al.  Residual Reconstruction for Block-Based Compressed Sensing of Video , 2011, 2011 Data Compression Conference.

[59]  Martin Vetterli,et al.  Compressive Sampling [From the Guest Editors] , 2008, IEEE Signal Processing Magazine.

[60]  Gary J. Sullivan,et al.  Multi-hypothesis motion compensation for low bit-rate video coding , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[61]  David V. Anderson,et al.  Compressive Sensing on a CMOS Separable-Transform Image Sensor , 2010, Proceedings of the IEEE.

[62]  Michel Barlaud,et al.  Image coding using wavelet transform , 1992, IEEE Trans. Image Process..

[63]  Thomas Maugey,et al.  Multistage compressed-sensing reconstruction of multiview images , 2010, 2010 IEEE International Workshop on Multimedia Signal Processing.

[64]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[65]  Michael T. Orchard,et al.  Overlapped block motion compensation: an estimation-theoretic approach , 1994, IEEE Trans. Image Process..

[66]  Wei Lu,et al.  Real-time dynamic MR image reconstruction using Kalman Filtered Compressed Sensing , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[67]  Justin K. Romberg,et al.  Compressive Sensing by Random Convolution , 2009, SIAM J. Imaging Sci..

[68]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[69]  Lawrence Carin,et al.  Tree-Structured Compressive Sensing With Variational Bayesian Analysis , 2010, IEEE Signal Processing Letters.

[70]  Laurent Jacques,et al.  Dequantizing Compressed Sensing: When Oversampling and Non-Gaussian Constraints Combine , 2009, IEEE Transactions on Information Theory.

[71]  Stephen J. Wright,et al.  Sparse reconstruction by separable approximation , 2009, IEEE Trans. Signal Process..

[72]  Richard G. Baraniuk,et al.  A new compressive imaging camera architecture using optical-domain compression , 2006, Electronic Imaging.

[73]  Lu Gan.  Block Compressed Sensing of Natural Images , 2007, 2007 15th International Conference on Digital Signal Processing.

[74]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[75]  Michael B. Wakin,et al.  A manifold lifting algorithm for multi-view compressive imaging , 2009, 2009 Picture Coding Symposium.

[76]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[77]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[78]  Thomas Wiegand,et al.  Long-term memory motion-compensated prediction , 1999, IEEE Trans. Circuits Syst. Video Technol..

[79]  Rebecca Willett,et al.  Compressive coded aperture imaging , 2009, Electronic Imaging.

[80]  Yonina C. Eldar,et al.  Foundations and Trends R © in Signal Processing , 2013 .

[81]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[82]  Trac D. Tran,et al.  Fast compressive sampling with structurally random matrices , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[83]  David S. Taubman,et al.  Lifting-based invertible motion adaptive transform (LIMAT) framework for highly scalable video compression , 2003, IEEE Trans. Image Process..

[84]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[85]  Arian Maleki,et al.  Optimally Tuned Iterative Reconstruction Algorithms for Compressed Sensing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[86]  Hayder Radha,et al.  On low bit-rate coding using the contourlet transform , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[87]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[88]  Bernd Girod,et al.  Efficiency analysis of multihypothesis motion-compensated prediction for video coding , 2000, IEEE Trans. Image Process..

[89]  Rama Chellappa,et al.  Compressive Acquisition of Dynamic Scenes , 2010, ECCV.

[90]  Chun-Shien Lu,et al.  Distributed compressive video sensing , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[91]  C. Guillemot,et al.  Distributed Monoview and Multiview Video Coding , 2007, IEEE Signal Processing Magazine.

[92]  Olgica Milenkovic,et al.  Information Theoretical and Algorithmic Approaches to Quantized Compressive Sensing , 2011, IEEE Transactions on Communications.

[93]  Jens-Rainer Ohm,et al.  Three-dimensional subband coding with motion compensation , 1994, IEEE Trans. Image Process..

[94]  Christine Guillemot,et al.  Image coding with iterated contourlet and wavelet transforms , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[95]  R. Olsson,et al.  High-Speed MEMS Micromirror Switching , 2007, 2007 Conference on Lasers and Electro-Optics (CLEO).

[96]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[97]  E.J. Candes.  Compressive Sampling , 2022 .

[98]  Wim Sweldens,et al.  Lifting scheme: a new philosophy in biorthogonal wavelet constructions , 1995, Optics + Photonics.

[99]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[100]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[101]  Jong Chul Ye,et al.  Motion estimated and compensated compressed sensing dynamic magnetic resonance imaging: What we can learn from video compression techniques , 2010, Int. J. Imaging Syst. Technol..

[102]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[103]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[104]  L. Landweber.  An iteration formula for Fredholm integral equations of the first kind , 1951 .

[105]  Philip Schniter,et al.  Fast Bayesian Matching Pursuit: Model Uncertainty and Parameter Estimation for Sparse Linear Models , 2009 .

[106]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[107]  John W. Woods,et al.  Bidirectional MC-EZBC with lifting implementation , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[108]  James E. Fowler,et al.  Video Compressed Sensing with Multihypothesis , 2011, 2011 Data Compression Conference.

[109]  Kameswara Namuduri,et al.  Distributed video coding for wireless sensor networks , 2005 .

[110]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[111]  Laurent Jacques,et al.  CMOS compressed imaging by Random Convolution , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[112]  Maria Trocan,et al.  Graph-Cut Rate Distortion Algorithm for Contourlet-Based Image Compression , 2007, 2007 IEEE International Conference on Image Processing.

[113]  Sanjoy Dasgupta,et al.  An elementary proof of a theorem of Johnson and Lindenstrauss , 2003, Random Struct. Algorithms.

[114]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[115]  J. Makhoul.  A fast cosine transform in one and two dimensions , 1980 .

[116]  James E. Fowler,et al.  Image coding using a complex dual-tree wavelet transform , 2007, 2007 15th European Signal Processing Conference.

[117]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[118]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[119]  I. Daubechies,et al.  Factoring wavelet transforms into lifting steps , 1998 .

[120]  Richard G. Baraniuk,et al.  Kronecker Compressive Sensing , 2012, IEEE Transactions on Image Processing.

[121]  Sundeep Rangan,et al.  On the Rate-Distortion Performance of Compressed Sensing , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[122]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[123]  James E. Fowler,et al.  Block Compressed Sensing of Images Using Directional Transforms , 2010, 2010 Data Compression Conference.

[124]  Hayder Radha,et al.  Wavelet-based contourlet transform and its application to image coding , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[125]  V.K. Goyal,et al.  Compressive Sampling and Lossy Compression , 2008, IEEE Signal Processing Magazine.

[126]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[127]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[128]  Dimitris Achlioptas,et al.  Database-friendly random projections: Johnson-Lindenstrauss with binary coins , 2003, J. Comput. Syst. Sci..

[129]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[130]  Richard G. Baraniuk,et al.  An Architecture for Compressive Imaging , 2006, 2006 International Conference on Image Processing.

[131]  Liang Xiao,et al.  Compressed sensing joint reconstruction for multi-view images , 2010 .

[132]  Itu-T and Iso Iec Jtc.  Advanced video coding for generic audiovisual services , 2010 .

[133]  Guangming Shi,et al.  Progressive Quantization of Compressive Sensing Measurements , 2011, 2011 Data Compression Conference.

[134]  Wei Lu,et al.  Modified compressive sensing for real-time dynamic MR imaging , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[135]  Wei Lu,et al.  Modified-CS: Modifying compressive sensing for problems with partially known support , 2009, 2009 IEEE International Symposium on Information Theory.

[136]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[137]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.