Generalized methods and solvers for noise removal from piecewise constant signals. II. New methods

Removing noise from signals which are piecewise constant (PWC) is a challenging signal processing problem that arises in many practical scientific and engineering contexts. In the first paper (part I) of this series of two, we presented background theory building on results from the image processing community to show that the majority of these algorithms, and more proposed in the wider literature, are each associated with a special case of a generalized functional, that, when minimized, solves the PWC denoising problem. It shows how the minimizer can be obtained by a range of computational solver algorithms. In this second paper (part II), using this understanding developed in part I, we introduce several novel PWC denoising methods, which, for example, combine the global behaviour of mean shift clustering with the local smoothing of total variation diffusion, and show example solver algorithms for these new methods. Comparisons between these methods are performed on synthetic and real signals, revealing that our new methods have a useful role to play. Finally, overlaps between the generalized methods of these two papers and others such as wavelet shrinkage, hidden Markov models, and piecewise smooth filtering are touched on.

[1]  Teuta Pilizota,et al.  A molecular brake, not a clutch, stops the Rhodobacter sphaeroides flagellar motor , 2009, Proceedings of the National Academy of Sciences.

[2]  Michio Homma,et al.  Direct observation of steps in rotation of the bacterial flagellar motor , 2005, Nature.

[3]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[4]  John W. Tukey,et al.  Exploratory Data Analysis. , 1979 .

[5]  Varit Chaisinthop,et al.  Semi-parametric compression of piecewise smooth functions , 2009, 2009 17th European Signal Processing Conference.

[6]  E. Rowan,et al.  Geologic Cross Section D-D' Through the Appalachian Basin from the Findlay Arch, Sandusky County, Ohio, to the Valley and Ridge Province, Hardy County, West Virginia , 2009 .

[7]  D. Donoho,et al.  Does median filtering truly preserve edges better than linear filtering , 2006, math/0612422.

[8]  Pin T. Ng,et al.  Quantile smoothing splines , 1994 .

[9]  Carlo Cattani,et al.  Haar wavelet-based technique for sharp jumps classification , 2004 .

[10]  Emmanuel J. Candès,et al.  Modern statistical estimation via oracle inequalities , 2006, Acta Numerica.

[11]  C. H. Mehta,et al.  Segmentation of well logs by maximum-likelihood estimation , 1990 .

[12]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[13]  Holger Hoefling A Path Algorithm for the Fused Lasso Signal Approximator , 2009, 0910.0526.

[14]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[15]  Petar M. Djuric,et al.  Automatic segmentation of piecewise constant signal by hidden Markov models , 1996, Proceedings of 8th Workshop on Statistical Signal and Array Processing.

[16]  Koen Visscher,et al.  An objective, model-independent method for detection of non-uniform steps in noisy signals , 2008, Comput. Phys. Commun..

[17]  Hal J. Bloom Next generation Geostationary Operational Environmental Satellite: GOES-R, the United States' advanced weather sentinel , 2009, Optical Engineering + Applications.

[18]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Stephan Didas,et al.  Splines in Higher Order TV Regularization , 2006, International Journal of Computer Vision.

[20]  Ursula Gather,et al.  Robust Detail‐Preserving Signal Extraction , 2005 .

[21]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[22]  Mark W. Schmidt,et al.  Fast Optimization Methods for L1 Regularization: A Comparative Study and Two New Approaches , 2007, ECML.

[23]  Jeff A. Bilmes,et al.  What HMMs Can Do , 2006, IEICE Trans. Inf. Syst..

[24]  Fabio Rocca,et al.  Modeling seismic impedance with Markov chains , 1980 .

[25]  R. Koenker Quantile Regression: Name Index , 2005 .

[26]  Regularization Paths for Least Squares Problems with Generalized $\ell_1$ Penalties , 2010 .

[27]  T. Chan,et al.  Edge-preserving and scale-dependent properties of total variation regularization , 2003 .

[28]  Ansgar Steland,et al.  On detecting jumps in time series: nonparametric setting , 2004 .

[29]  A. Iserles A First Course in the Numerical Analysis of Differential Equations: Stiff equations , 2008 .

[30]  Jérôme Darbon,et al.  Image Restoration with Discrete Constrained Total Variation Part I: Fast and Exact Optimization , 2006, Journal of Mathematical Imaging and Vision.

[31]  Stephen P. Boyd,et al.  1 Trend Filtering , 2009, SIAM Rev..

[32]  Emmanuel J. Candès,et al.  New multiscale transforms, minimum total variation synthesis: applications to edge-preserving image reconstruction , 2002, Signal Process..

[33]  Roland Fried,et al.  On the robust detection of edges in time series filtering , 2007, Comput. Stat. Data Anal..

[34]  S. Mallat A wavelet tour of signal processing , 1998 .

[35]  P. Prandoni Optimal segmentation techniques for piecewise stationary signals , 1999 .

[36]  Ajay N. Jain,et al.  Assembly of microarrays for genome-wide measurement of DNA copy number , 2001, Nature Genetics.

[37]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[38]  R. A. Kennedy,et al.  Forward-backward non-linear filtering technique for extracting small biological signals from noise , 1991, Journal of Neuroscience Methods.

[39]  Thomas Brox,et al.  On the Equivalence of Soft Wavelet Shrinkage, Total Variation Diffusion, Total Variation Regularization, and SIDEs , 2004, SIAM J. Numer. Anal..

[40]  S. McKinney,et al.  Analysis of single-molecule FRET trajectories using hidden Markov modeling. , 2006, Biophysical journal.

[41]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[43]  Tony F. Chan,et al.  Image processing and analysis - variational, PDE, wavelet, and stochastic methods , 2005 .

[44]  J. Suykens,et al.  Convex Clustering Shrinkage , 2005 .

[45]  Shin Ta Liu,et al.  Nonlinear Signal Processing: A Statistical Approach , 2006, Technometrics.

[46]  E. S. Page A test for a change in a parameter occurring at an unknown point , 1955 .

[47]  Stéphane Mallat,et al.  Singularity detection and processing with wavelets , 1992, IEEE Trans. Inf. Theory.

[48]  D. Gill Application of a Statistical Zonation Method to Reservoir Evaluation and Digitized-Log Analysis , 1970 .

[49]  S H Chung,et al.  Characterization of single channel currents using digital signal processing techniques based on Hidden Markov Models. , 1990, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[50]  Yazhen Wang Jump and sharp cusp detection by wavelets , 1995 .

[51]  J. Franklin,et al.  The elements of statistical learning: data mining, inference and prediction , 2005 .

[52]  Christy F Landes,et al.  Denoising single-molecule FRET trajectories with wavelets and Bayesian inference. , 2010, Biophysical journal.

[53]  Michael Elad,et al.  On the origin of the bilateral filter and ways to improve it , 2002, IEEE Trans. Image Process..

[54]  Gonzalo R. Arce,et al.  Nonlinear Signal Processing - A Statistical Approach , 2004 .

[55]  P. Mrázek,et al.  ON ROBUST ESTIMATION AND SMOOTHING WITH SPATIAL AND TONAL KERNELS , 2006 .

[56]  S. Rosset,et al.  Piecewise linear regularized solution paths , 2007, 0708.2197.

[57]  Jérôme Darbon,et al.  Image Restoration with Discrete Constrained Total Variation Part II: Levelable Functions, Convex Priors and Non-Convex Cases , 2006, Journal of Mathematical Imaging and Vision.

[58]  Klaus Nordhausen,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition by Trevor Hastie, Robert Tibshirani, Jerome Friedman , 2009 .

[59]  Liedewij Laan,et al.  Assembly dynamics of microtubules at molecular resolution , 2006, Nature.

[60]  Stéphane Mallat,et al.  Characterization of Signals from Multiscale Edges , 2011, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  R. Tibshirani,et al.  PATHWISE COORDINATE OPTIMIZATION , 2007, 0708.1485.

[62]  B I Justusson,et al.  Median Filtering: Statistical Properties , 1981 .

[63]  Zoubin Ghahramani,et al.  A Unifying Review of Linear Gaussian Models , 1999, Neural Computation.

[64]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[65]  Jean Serra,et al.  Image Analysis and Mathematical Morphology , 1983 .

[66]  Raymond H. Chan,et al.  A Detection Statistic for Random-Valued Impulse Noise , 2007, IEEE Transactions on Image Processing.

[67]  J CandèsEmmanuel,et al.  New multiscale transforms, minimum total variation synthesis , 2002 .

[68]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .