Consistencies and rates of convergence of jump-penalized least squares estimators

We study the asymptotics for jump-penalized least squares regression aiming at approximating a regression function by piecewise constant functions. Besides conventional consistency and convergence rates of the estimates in L 2 ([0,1)) our results cover other metrics like Skorokhod metric on the space of cadlag functions and uniform metrics on C([0, 1]). We will show that these estimators are in an adaptive sense rate optimal over certain classes of "approximation spaces." Special cases are the class of functions of bounded variation (piecewise) Holder continuous functions of order 0 < α ≤ 1 and the class of step functions with a finite but arbitrary number of jumps. In the latter setting, we will also deduce the rates known from change-point analysis for detecting the jumps. Finally, the issue of fully automatic selection of the smoothing parameter is addressed.

[1]  R. Bass.  Convergence of probability measures , 2011 .

[2]  G. Winkler,et al.  Complexity Penalized M-Estimation , 2008 .

[3]  H. Leeb,et al.  Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator , 2007, 0704.1466.

[4]  P. Massart,et al.  Minimal Penalties for Gaussian Model Selection , 2007 .

[5]  Axel Munk,et al.  Scale space consistency of piecewise constant least squares estimators - another look at the regressogram , 2006, math/0609347.

[6]  D. Donoho.  For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[7]  G. Winkler,et al.  Complexity Penalised M-Estimation: Fast Computation , 2005 .

[8]  Felix Friedrich.  Complexity Penalized Segmentations in 2D , 2005 .

[9]  Felix Friedrich,et al.  Beyond wavelets: New image representation paradigms , 2005 .

[10]  Olaf Wittich,et al.  Don't shed tears over breaks , 2005 .

[11]  A. Kempe.  Statistical Analysis of Discontinuous Phenomena with Potts Functionals , 2004 .

[12]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[13]  J. Polzehl,et al.  Image denoising: Pointwise adaptive approach , 2003 .

[14]  G. Winkler,et al.  Smoothers for Discontinuous Signals , 2002 .

[15]  Andrea Braides.  Γ-convergence for beginners , 2002 .

[16]  James Stephen Marron,et al.  Presentation of smoothers: the family approach , 2001, Comput. Stat..

[17]  P. Davies,et al.  Local Extremes, Runs, Strings and Multiresolution , 2001 .

[18]  Saad T. Bakir,et al.  Nonparametric Regression and Spline Smoothing , 2000, Technometrics.

[19]  H. Müller,et al.  Multiple changepoint fitting via quasilikelihood, with application to DNA sequence segmentation , 2000 .

[20]  J. Marron,et al.  SCALE SPACE VIEW OF CURVE ESTIMATION , 2000 .

[21]  J. Marron,et al.  SiZer for Exploration of Structures in Curves , 1999 .

[22]  D. Donoho.  Wedgelets: nearly minimax estimation of edges , 1999 .

[23]  M. Kohler.  Nonparametric estimation of piecewise smooth regression functions , 1999 .

[24]  H. Müller,et al.  Discontinuous versus smooth regression , 1999 .

[25]  J. Marron,et al.  Edge-Preserving Smoothers for Image Processing , 1998 .

[26]  R. DeVore,et al.  Nonlinear approximation , 1998, Acta Numerica.

[27]  Vladimir Spokoiny,et al.  ESTIMATION OF A FUNCTION WITH DISCONTINUITIES VIA LOCAL POLYNOMIAL FIT WITH AN ADAPTIVE WINDOW CHOICE , 1998 .

[28]  D. Donoho.  CART AND BEST-ORTHO-BASIS: A CONNECTION' , 1997 .

[29]  S. Geer,et al.  Locally adaptive regression splines , 1997 .

[30]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[31]  Fred Godtliebsen,et al.  A nonlinear gaussian filter applied to images with discontinuities , 1997 .

[32]  C. Loader.  CHANGE POINT ESTIMATION USING NONPARAMETRIC REGRESSION , 1996 .

[33]  Christian Hess,et al.  Epi-convergence of sequences of normal integrands and strong consistency of the maximum likelihood estimator , 1996 .

[34]  Jette Christensen,et al.  Multiple change-point analysis of disease incidence rates , 1996 .

[35]  V. G. Weierstrass,et al.  Estimation of a function with discontinuities via local polynomial fit with an adaptive window choice , 1996 .

[36]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[37]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[38]  Jörg Weule,et al.  Non-Linear Gaussian Filters Performing Edge Preserving Diffusion , 1995, DAGM-Symposium.

[39]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[40]  Q. Shao,et al.  On a conjecture of Revesz , 1995 .

[41]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[42]  I. Johnstone,et al.  Minimax risk overlp-balls forlp-error , 1994 .

[43]  H. Künsch.  Robust priors for smoothing and image restoration , 1994 .

[44]  I. Johnstone,et al.  Minimax Risk over l p-Balls for l q-error , 1994 .

[45]  Gerald Beer,et al.  Topologies on Closed and Closed Convex Sets , 1993 .

[46]  George G. Lorentz,et al.  Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[47]  G. D. Maso,et al.  An Introduction to-convergence , 1993 .

[48]  P. Hall,et al.  Edge-preserving and peak-preserving smoothing , 1992 .

[49]  M. Wand,et al.  EXACT MEAN INTEGRATED SQUARED ERROR , 1992 .

[50]  H. Müller.  CHANGE-POINTS IN NONPARAMETRIC REGRESSION ANALYSIS' , 1992 .

[51]  D R Fredkin,et al.  Bayesian restoration of single-channel patch clamp recordings. , 1992, Biometrics.

[52]  A. Shiryayev.  On Sums of Independent Random Variables , 1992 .

[53]  M. C. Jones,et al.  A reliable data-based bandwidth selection method for kernel density estimation , 1991 .

[54]  Yi-Ching Yao.  Estimating the number of change-points via Schwarz' criterion , 1988 .

[55]  S. Panchapakesan,et al.  Inference about the Change-Point in a Sequence of Random Variables: A Selection Approach , 1988 .

[56]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  G. Matheron.  Random Sets and Integral Geometry , 1976 .

[58]  V. V. Petrov,et al.  Sums of Independent Random Variables , 1976 .

[59]  H. G. Burchard,et al.  Piecewise polynomial approximation on optimal meshes , 1975 .

[60]  R. Tomkins.  On the law of the iterated logarithm for double sequences of random variables , 1974 .

[61]  Piecewise polynomial approximation , 1972 .

[62]  P. Billingsley,et al.  Convergence of Probability Measures , 1970, The Mathematical Gazette.

[63]  David V. Hinkley,et al.  Inference about the change-point in a sequence of binomial variables , 1970 .

[64]  J. Sacks,et al.  Designs for Regression Problems with Correlated Errors III , 1966 .

[65]  J. Tukey.  Curves As Parameters, and Touch Estimation , 1961 .

[66]  E. Ising.  Beitrag zur Theorie des Ferromagnetismus , 1925 .