Sparsity Based Methods for Overparametrized Variational Problems

Two complementary approaches have been extensively used in signal and image processing leading to novel results, the sparse representation methodology and the variational strategy. Recently, a new sparsity based model has been proposed, the cosparse analysis framework, which may potentially help in bridging sparse approximation based methods to the traditional total-variation minimization. Based on this, we introduce a sparsity based framework for solving overparameterized variational problems. The latter has been used to improve the estimation of optical flow and also for general denoising of signals and images. However, the recovery of the space varying parameters involved was not adequately addressed by traditional variational methods. We first demonstrate the efficiency of the new framework for one dimensional signals in recovering a piecewise linear and polynomial function. Then, we illustrate how the new technique can be used for denoising and segmentation of images.

[1]  Minh N. Do,et al.  A Theory for Sampling Signals from a Union of Subspaces , 2022 .

[2]  Michael Elad,et al.  On the stability of the basis pursuit in the presence of noise , 2006, Signal Process..

[3]  Michael Elad,et al.  Cosamp and SP for the cosparse analysis model , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[4]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[5]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[6]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  T. Brox,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik a Survey on Variational Optic Flow Methods for Small Displacements a Survey on Variational Optic Flow Methods for Small Displacements , 2022 .

[8]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[9]  Michael Elad,et al.  The Cosparse Analysis Model and Algorithms , 2011, ArXiv.

[10]  Michael Elad,et al.  Can we allow linear dependencies in the dictionary in the sparse synthesis framework? , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions , 1994 .

[12]  Michael Elad,et al.  RIP-Based Near-Oracle Performance Guarantees for SP, CoSaMP, and IHT , 2012, IEEE Transactions on Signal Processing.

[13]  D. Jupp Approximation to Data by Splines with Free Knots , 1978 .

[14]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[15]  Alfred M. Bruckstein,et al.  Over-Parameterized Optical Flow Using a Stereoscopic Constraint , 2011, SSVM.

[16]  Michael Elad,et al.  OMP with Highly Coherent Dictionaries , 2013 .

[17]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[18]  Raja Giryes,et al.  Greedy Signal Space Methods for incoherence and beyond , 2013, ArXiv.

[19]  Rachel Ward,et al.  Stable and Robust Sampling Strategies for Compressive Imaging , 2012, IEEE Transactions on Image Processing.

[20]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[21]  Michael Elad,et al.  Performance Guarantees of the Thresholding Algorithm for the Cosparse Analysis Model , 2013, IEEE Transactions on Information Theory.

[22]  Simon Foucart,et al.  Hard Thresholding Pursuit: An Algorithm for Compressive Sensing , 2011, SIAM J. Numer. Anal..

[23]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[24]  Pierre Kornprobst,et al.  Mathematical problems in image processing - partial differential equations and the calculus of variations , 2010, Applied mathematical sciences.

[25]  Raja Giryes,et al.  Near oracle performance and block analysis of signal space greedy methods , 2014, J. Approx. Theory.

[26]  Yonina C. Eldar,et al.  Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors , 2008, IEEE Transactions on Signal Processing.

[27]  Bhaskar D. Rao,et al.  An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem , 2007, IEEE Transactions on Signal Processing.

[28]  Raja Giryes,et al.  Near Oracle Performance of Signal Space Greedy Methods , 2014, ArXiv.

[29]  M. Davies,et al.  Greedy-like algorithms for the cosparse analysis model , 2012, 1207.2456.

[30]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[31]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[32]  Alfred M. Bruckstein,et al.  Over-Parameterized Variational Optical Flow , 2007, International Journal of Computer Vision.

[33]  L. Ambrosio,et al.  Approximation of functional depending on jumps by elliptic functional via t-convergence , 1990 .

[34]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[35]  Michael Elad,et al.  Recovery of cosparse signals with Greedy Analysis Pursuit in the presence of noise , 2011, 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[36]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[37]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[38]  Steven Kay,et al.  Optimal segmentation of signals and its application to image denoising and boundary feature extraction , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[39]  Joachim Weickert,et al.  Anisotropic diffusion in image processing , 1996 .

[40]  Arjan Kuijper,et al.  Scale Space and Variational Methods in Computer Vision , 2013, Lecture Notes in Computer Science.

[41]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[42]  Deanna Needell,et al.  Signal Space CoSaMP for Sparse Recovery With Redundant Dictionaries , 2012, IEEE Transactions on Information Theory.

[43]  Emmanuel J. Candès,et al.  Towards a Mathematical Theory of Super‐resolution , 2012, ArXiv.

[44]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .

[45]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[46]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[47]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[48]  Emmanuel J. Cand The Restricted Isometry Property and Its Implications for Compressed Sensing , 2008 .

[49]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[50]  Jean-Michel Morel,et al.  Variational methods in image segmentation , 1995 .

[51]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[52]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[53]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[54]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[55]  智一 吉田,et al.  Efficient Graph-Based Image Segmentationを用いた圃場図自動作成手法の検討 , 2014 .

[56]  Deanna Needell,et al.  Using Correlated Subset Structure for Compressive Sensing Recovery , 2013, ArXiv.

[57]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[58]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[59]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[60]  Emmanuel J. Candès,et al.  Modern statistical estimation via oracle inequalities , 2006, Acta Numerica.

[61]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[62]  Tony F. Chan,et al.  Image processing and analysis , 2005 .

[63]  Mohamed-Jalal Fadili,et al.  Robust Sparse Analysis Regularization , 2011, IEEE Transactions on Information Theory.

[64]  Alfred M. Bruckstein,et al.  On Globally Optimal Local Modeling: From Moving Least Squares to Over-parametrization , 2013, Innovations for Shape Analysis, Models and Algorithms.

[65]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[66]  Ke Chen,et al.  On Multigrids for Solving a Class of Improved Total Variation Based Staircasing Reduction Models , 2007 .

[67]  D. Donoho Sparse Components of Images and Optimal Atomic Decompositions , 2001 .

[68]  Rémi Gribonval,et al.  Iterative cosparse projection algorithms for the recovery of cosparse vectors , 2011, 2011 19th European Signal Processing Conference.

[69]  J. Tropp Algorithms for simultaneous sparse approximation. Part II: Convex relaxation , 2006, Signal Process..

[70]  Massimo Fornasier,et al.  Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints , 2008, SIAM J. Numer. Anal..

[71]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[72]  Deanna Needell,et al.  Stable Image Reconstruction Using Total Variation Minimization , 2012, SIAM J. Imaging Sci..

[73]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[74]  A. Bruckstein,et al.  On Over-Parameterized Model Based TV-Denoising , 2007, 2007 International Symposium on Signals, Circuits and Systems.