Audio Inpainting: Revisited and Reweighted

In this article, we deal with the problem of sparsity-based audio inpainting, i.e. filling in the missing segments of audio. A consequence of the approaches based on mathematical optimization is the insufficient amplitude of the signal in the filled gaps. Remaining in the framework based on sparsity and convex optimization, we propose improvements to audio inpainting, aiming at compensating for such an energy loss. The new ideas are based on different types of weighting, both in the coefficient and the time domains. We show that our propositions improve the inpainting performance in terms of both the SNR and ODG.

[1]  Hans-Georg Stark,et al.  Audio inpainting: Evaluation of time-frequency representations and structured sparsity approaches , 2018, Signal Process..

[2]  Ondrej Mokrý,et al.  Introducing SPAIN (SParse Audio INpainter) , 2018, 2019 27th European Signal Processing Conference (EUSIPCO).

[3]  Ondrej Mokrý,et al.  A Proper Version of Synthesis-based Sparse Audio Declipper , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Nathanael Perraudin,et al.  A Context Encoder For Audio Inpainting , 2018, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[5]  Nathanael Perraudin,et al.  Audio Inpainting of Music by Means of Neural Networks , 2018, 1810.12138.

[6]  Laurent Oudre,et al.  Interpolation of Missing Samples in Sound Signals Based on Autoregressive Modeling , 2018, Image Process. Line.

[7]  Pavel Záviska,et al.  Revisiting Synthesis Model in Sparse Audio Declipper , 2018, LVA/ICA.

[8]  Mark D. Plumbley,et al.  Consistent Dictionary Learning for Signal Declipping , 2018, LVA/ICA.

[9]  Nicki Holighaus,et al.  Inpainting of Long Audio Segments With Similarity Graphs , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[10]  Patrick Pérez,et al.  Audio declipping via nonnegative matrix factorization , 2015, 2015 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA).

[11]  Nicki Holighaus,et al.  Acceleration of audio inpainting by support restriction , 2015, 2015 7th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT).

[12]  Rémi Gribonval,et al.  Sparsity and Cosparsity for Audio Declipping: A Flexible Non-convex Approach , 2015, LVA/ICA.

[13]  Michael Elad,et al.  Self-content-based audio inpainting , 2015, Signal Process..

[14]  J. Pesquet,et al.  Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems , 2014, IEEE Signal Processing Magazine.

[15]  Laurent Condat,et al.  A Generic Proximal Algorithm for Convex Optimization—Application to Total Variation Minimization , 2014, IEEE Signal Processing Letters.

[16]  Nicki Holighaus,et al.  The Large Time-Frequency Analysis Toolbox 2.0 , 2013, CMMR.

[17]  Hans G. Feichtinger,et al.  Advances in Gabor Analysis , 2012 .

[18]  Michael Elad,et al.  Audio Inpainting , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  Michael B. Wakin,et al.  Recovering a Clipped Signal in Sparseland , 2011, ArXiv.

[20]  Michael Elad,et al.  A constrained matching pursuit approach to audio declipping , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[21]  Michael Elad Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[22]  Stephen P. Boyd,et al.  Convex Optimization , 2004, IEEE Transactions on Automatic Control.

[23]  Massimo Fornasier,et al.  Theoretical Foundations and Numerical Methods for Sparse Recovery , 2010, Radon Series on Computational and Applied Mathematics.

[24]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[25]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[26]  O. Christensen Frames and Bases: An Introductory Course , 2008 .

[27]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[28]  Birger Kollmeier,et al.  PEMO-Q—A New Method for Objective Audio Quality Assessment Using a Model of Auditory Perception , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[29]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[30]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[31]  Mathieu Lagrange,et al.  Long Interpolation of Audio Signals Using Linear Prediction in Sinusoidal Modeling , 2005 .

[32]  Pavel Rajmic,et al.  Exact risk analysis of wavelet spectrum thresholding rules , 2003, 10th IEEE International Conference on Electronics, Circuits and Systems, 2003. ICECS 2003. Proceedings of the 2003.

[33]  D. Donoho,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[34]  O. Christensen An introduction to frames and Riesz bases , 2002 .

[35]  P. Hedelin,et al.  Packet loss concealment based on sinusoidal modeling , 2002, Speech Coding, 2002, IEEE Workshop Proceedings..

[36]  Jyrki Kauppinen,et al.  Reconstruction Method for Missing or Damaged Long Portions in Audio Signal , 2002 .

[37]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[38]  Peter J. W. Rayner,et al.  Digital Audio Restoration: A Statistical Model Based Approach , 1998 .

[39]  W. Etter,et al.  Restoration of a discrete-time signal segment by interpolation based on the left-sided and right-sided autoregressive parameters , 1996, IEEE Trans. Signal Process..

[40]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[41]  Raymond N. J. Veldhuis,et al.  Adaptive interpolation of discrete-time signals that can be modeled as autoregressive processes , 1986, IEEE Trans. Acoust. Speech Signal Process..

[42]  Pavel Rajmic,et al.  Reweighted ` 1 minimization for audio inpainting , 2019 .

[43]  Yiannis Kompatsiaris,et al.  Sparse audio inpainting with variational Bayesian inference , 2018, 2018 IEEE International Conference on Consumer Electronics (ICCE).

[44]  Christina Gloeckner Foundations Of Time Frequency Analysis , 2016 .

[45]  Moni Naor,et al.  Latent Variable Analysis and Signal Separation , 2015, Lecture Notes in Computer Science.

[46]  A. Chambolle,et al.  A rst-order primal-dual algorithm for convex problems with applications to imaging , 2010 .

[47]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[48]  Vesa Välimäki,et al.  Interpolation of Long Gaps in Audio Signals Using the Warped Burg's Method , 2003 .

[49]  Ismo Kauppinen,et al.  Audio Signal Extrapolation - Theory And Applications , 2002 .

[50]  J. Ponssard,et al.  ECOLE POLYTECHNIQUE , 1994 .

[51]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[52]  S. Geneva,et al.  Sound Quality Assessment Material: Recordings for Subjective Tests , 1988 .

[53]  J. Moreau Proximité et dualité dans un espace hilbertien , 1965 .