High-Accuracy Total Variation With Application to Compressed Video Sensing

Numerous total variation (TV) regularizers, engaged in image restoration problem, encode the gradients by means of simple [-1, 1] finite-impulse-response (FIR) filter. Despite its low computational processing, this filter severely distorts signal's high-frequency components pertinent to edge/ discontinuous information and cause several deficiency issues known as texture and geometric loss. This paper addresses this problem by proposing an alternative model to the TV regularization problem via high-order accuracy differential FIR filters to preserve rapid transitions in signal recovery. A numerical encoding scheme is designed to extend the TV model into multidimensional representation (tensorial decomposition). We adopt this design to regulate the spatial and temporal redundancy in compressed video sensing problem to jointly recover frames from undersampled measurements. We then seek the solution via alternating direction methods of multipliers and find a unique solution to quadratic minimization step with capability of handling different boundary conditions. The resulting algorithm uses much lower sampling rate and highly outperforms alternative state-of-the-art methods. This is evaluated both in terms of restoration accuracy and visual quality of the recovered frames.

[1]  Alan J. Laub,et al.  Matrix analysis - for scientists and engineers , 2004 .

[2]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[3]  G. Styan,et al.  Equalities and Inequalities for Ranks of Matrices , 1974 .

[4]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[5]  Raymond H. Chan,et al.  A Fast Algorithm for Deblurring Models with Neumann Boundary Conditions , 1999, SIAM J. Sci. Comput..

[6]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[7]  Ofer Levi,et al.  Optical compressive change and motion detection. , 2012, Applied optics.

[8]  Jong Chul Ye,et al.  k‐t FOCUSS: A general compressed sensing framework for high resolution dynamic MRI , 2009, Magnetic resonance in medicine.

[9]  A. Robert Calderbank,et al.  A fast reconstruction algorithm for deterministic compressive sensing using second order reed-muller codes , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[10]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[11]  Bernd Jähne,et al.  Spatio-Temporal Image Processing , 1993, Lecture Notes in Computer Science.

[12]  L. Ambrosio,et al.  Functions of Bounded Variation and Free Discontinuity Problems , 2000 .

[13]  Michael B. Wakin,et al.  A multiscale framework for Compressive Sensing of video , 2009, 2009 Picture Coding Symposium.

[14]  Jahn Müller,et al.  Higher-Order TV Methods—Enhancement via Bregman Iteration , 2012, Journal of Scientific Computing.

[15]  David D. Wentzloff,et al.  1.2 GS/s Hadamard Transform front-end for compressive sensing in 65nm CMOS , 2013, 2013 IEEE Radio and Wireless Symposium.

[16]  Dimitri Van De Ville,et al.  A Signal Processing Approach to Generalized 1-D Total Variation , 2011, IEEE Transactions on Signal Processing.

[17]  Daniel Cremers,et al.  Anisotropic Huber-L1 Optical Flow , 2009, BMVC.

[18]  LiLiyuan,et al.  Statistical modeling of complex backgrounds for foreground object detection , 2004 .

[19]  Timothy J. McDevitt,et al.  Discrete Lanczos derivatives of noisy data , 2012, Int. J. Comput. Math..

[20]  Benar Fux Svaiter,et al.  General Projective Splitting Methods for Sums of Maximal Monotone Operators , 2009, SIAM J. Control. Optim..

[21]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[22]  Didier Le Gall,et al.  MPEG: a video compression standard for multimedia applications , 1991, CACM.

[23]  B. Fornberg Generation of finite difference formulas on arbitrarily spaced grids , 1988 .

[24]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[25]  Andy M. Yip,et al.  Total Variation Image Restoration: Overview and Recent Developments , 2006, Handbook of Mathematical Models in Computer Vision.

[26]  Chengbo Li An efficient algorithm for total variation regularization with applications to the single pixel camera and compressive sensing , 2010 .

[27]  Michael B. Wakin,et al.  Multiscale algorithm for reconstructing videos from streaming compressive measurements , 2013, J. Electronic Imaging.

[28]  B. J Hne,et al.  Spatio - temporal Image Processing: Theory and Scientific Applications , 1991 .

[29]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[30]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[31]  Ronald F. Gariepy FUNCTIONS OF BOUNDED VARIATION AND FREE DISCONTINUITY PROBLEMS (Oxford Mathematical Monographs) , 2001 .

[32]  ChanTony,et al.  High-Order Total Variation-Based Image Restoration , 2000 .

[33]  ITU-T Rec. P.910 (04/2008) Subjective video quality assessment methods for multimedia applications , 2009 .

[34]  Ming Li,et al.  Motion-Aware Decoding of Compressed-Sensed Video , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Konstantinos N. Plataniotis,et al.  Temporal derivatives in compressed video sensing , 2012, 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP).

[36]  A. Wathen,et al.  Iterative Methods for Toeplitz Systems , 2005 .

[37]  Thomas Brox,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Highly Accurate Optic Flow Computation with Theoretically Justified Warping Highly Accurate Optic Flow Computation with Theoretically Justified Warping , 2022 .

[38]  Abbas El Gamal,et al.  CMOS Image Sensor With Per-Column ΣΔ ADC and Programmable Compressed Sensing , 2013, IEEE Journal of Solid-State Circuits.

[39]  J. Spingarn Partial inverse of a monotone operator , 1983 .

[40]  Qi Tian,et al.  Statistical modeling of complex backgrounds for foreground object detection , 2004, IEEE Transactions on Image Processing.

[41]  Stefano Serra Capizzano,et al.  A Note on Antireflective Boundary Conditions and Fast Deblurring Models , 2004, SIAM J. Sci. Comput..

[42]  Joachim Weickert,et al.  Variational Optic Flow Computation with a Spatio-Temporal Smoothness Constraint , 2001, Journal of Mathematical Imaging and Vision.

[43]  Jianping Li General explicit difference formulas for numerical differentiation , 2005 .

[44]  R. Ohba,et al.  New finite difference formulas for numerical differentiation , 2000 .

[45]  Narendra Ahuja,et al.  Imaging via three-dimensional compressive sampling (3DCS) , 2011, 2011 International Conference on Computer Vision.

[46]  Aswin C. Sankaranarayanan,et al.  CS-MUVI: Video compressive sensing for spatial-multiplexing cameras , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[47]  S. Tubaro,et al.  Subjective assessment of H.264/AVC video sequences transmitted over a noisy channel , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[48]  Jeffrey A. Fessler,et al.  Accelerated Edge-Preserving Image Restoration Without Boundary Artifacts , 2013, IEEE Transactions on Image Processing.

[49]  Rosemary A. Renaut,et al.  Improved Total Variation-Type Regularization Using Higher Order Edge Detectors , 2010, SIAM J. Imaging Sci..

[50]  Karl Kunisch,et al.  Total Generalized Variation , 2010, SIAM J. Imaging Sci..

[51]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[52]  Richard G. Baraniuk,et al.  Kronecker Compressive Sensing , 2012, IEEE Transactions on Image Processing.

[53]  Truong Q. Nguyen,et al.  An Augmented Lagrangian Method for Total Variation Video Restoration , 2011, IEEE Transactions on Image Processing.

[54]  Abbas El Gamal,et al.  Optical flow estimation using temporally oversampled video , 2005, IEEE Transactions on Image Processing.

[55]  Mário A. T. Figueiredo,et al.  Deconvolving Images With Unknown Boundaries Using the Alternating Direction Method of Multipliers , 2012, IEEE Transactions on Image Processing.

[56]  Junfeng Yang,et al.  An Efficient TVL1 Algorithm for Deblurring Multichannel Images Corrupted by Impulsive Noise , 2009, SIAM J. Sci. Comput..

[57]  Rebecca Willett,et al.  Compressive coded aperture video reconstruction , 2008, 2008 16th European Signal Processing Conference.

[58]  Abbas El Gamal,et al.  A 256×256 CMOS image sensor with ΔΣ-based single-shot compressed sensing , 2012, 2012 IEEE International Solid-State Circuits Conference.

[59]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[60]  S. Lele Compact finite difference schemes with spectral-like resolution , 1992 .

[61]  Enrico Magli,et al.  A parallel compressive imaging architecture for one-shot acquisition , 2013, 2013 Picture Coding Symposium (PCS).

[62]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[63]  Daniel Cremers,et al.  Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation , 2005, International Journal of Computer Vision.

[64]  Shahram Shirani,et al.  Compressive Sensing Image Sensors-Hardware Implementation , 2013, Sensors.

[65]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[66]  M. Nikolova An Algorithm for Total Variation Minimization and Applications , 2004 .

[67]  M. Salman Asif,et al.  Motion‐adaptive spatio‐temporal regularization for accelerated dynamic MRI , 2013, Magnetic resonance in medicine.

[68]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[69]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[70]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[71]  Richard Lippmann,et al.  Review of Neural Networks for Speech Recognition , 1989, Neural Computation.

[72]  Tony F. Chan,et al.  High-Order Total Variation-Based Image Restoration , 2000, SIAM J. Sci. Comput..

[73]  Shiqian Ma,et al.  Fast Multiple-Splitting Algorithms for Convex Optimization , 2009, SIAM J. Optim..

[74]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[75]  I. R. Khan,et al.  Closed-form expressions for the finite approximations of first and higher derivatives based on Taylor series , 1999 .

[76]  Yin Zhang,et al.  Scalable video coding using compressive sensing , 2012, Bell Labs Technical Journal.

[77]  Jae Wook Kim,et al.  High-order compact filters with variable cut-off wavenumber and stable boundary treatment , 2010 .

[78]  Andrzej Cichocki,et al.  Computing Sparse Representations of Multidimensional Signals Using Kronecker Bases , 2013, Neural Computation.

[79]  José M. Bioucas-Dias,et al.  Restoration of Poissonian Images Using Alternating Direction Optimization , 2010, IEEE Transactions on Image Processing.

[80]  Pierre Vandergheynst,et al.  Compressive image acquisition in modern CMOS IC design , 2015, Int. J. Circuit Theory Appl..

[81]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[82]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .