A geometrical approach to Iterative Isotone Regression

In the present paper, we propose and analyze a novel method for estimating a univariate regression function of bounded variation. The underpinning idea is to combine two classical tools in nonparametric statistics, namely isotonic regression and the estimation of additive models. A geometrical interpretation enables us to link this iterative method with Von Neumann's algorithm. Moreover, making a connection with the general property of isotonicity of projection onto convex cones, we derive another equivalent algorithm and go further in the analysis.

[1]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[2]  F. T. Wright.  The Asymptotic Behavior of Monotone Regression Estimates , 1981 .

[3]  Enno Mammen,et al.  Additive isotone regression , 2007, 0709.0888.

[4]  C. Durot,et al.  On the $\mathbb{L}_p$-error of monotonicity constrained estimators , 2007, 0708.2219.

[5]  Nicolas W. Hengartner,et al.  Iterative bias reduction: a comparative study , 2013, Stat. Comput..

[6]  Heinz H. Bauschke,et al.  Dykstra's Alternating Projection Algorithm for Two Sets , 1994 .

[7]  N. Hengartner,et al.  Recursive bias estimation for multivariate regression smoothers , 2011 .

[8]  Michael J. Best,et al.  Active set algorithms for isotonic regression; A unifying framework , 1990, Math. Program..

[9]  P. Bühlmann,et al.  Boosting With the L2 Loss , 2003 .

[10]  Nicolas Jégou.  Régression isotonique itérée , 2012 .

[11]  Clifford M. Hurvich,et al.  Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion , 1998 .

[12]  H. D. Brunk,et al.  Statistical inference under order restrictions : the theory and application of isotonic regression , 1973 .

[13]  G. Wahba.  Smoothing noisy data with spline functions , 1975 .

[14]  Chu-in Charles Lee,et al.  The Min-Max Algorithm and Isotonic Regression , 1983 .

[15]  Mary C. Meyer,et al.  ON THE DEGREES OF FREEDOM IN SHAPE-RESTRICTED REGRESSION , 2000 .

[16]  H. D. Brunk,et al.  AN EMPIRICAL DISTRIBUTION FUNCTION FOR SAMPLING WITH INCOMPLETE INFORMATION , 1955 .

[17]  Sandor Nemeth,et al.  Iterative methods for nonlinear complementarity problems on isotone projection cones , 2009 .

[18]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[19]  R. Dykstra.  An Algorithm for Restricted Least Squares Regression , 1983 .

[20]  G. Isac,et al.  Monotonicity of metric projections onto positive cones of ordered Euclidean spaces , 1986 .

[21]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[22]  Charles C. Taylor,et al.  On boosting kernel regression , 2008 .

[23]  Jon C. Dattorro,et al.  Convex Optimization & Euclidean Distance Geometry , 2004 .

[24]  Emmanuel Barillot,et al.  Analysis of array CGH data: from signal ratio to gain and loss of DNA regions , 2004, Bioinform..

[25]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[26]  A. B. Németh,et al.  How to project onto an isotone projection cone , 2010 .

[27]  Richard L. Dykstra.  An isotonic regression algorithm , 1981 .

[28]  P. Bühlmann,et al.  Boosting with the L2-loss: regression and classification , 2001 .

[29]  P. Hall,et al.  On the backfitting algorithm for additive regression models , 1993 .

[30]  H. D. Brunk.  Maximum Likelihood Estimates of Monotone Parameters , 1955 .

[31]  David Ruppert,et al.  Fitting a Bivariate Additive Model by Local Polynomial Regression , 1997 .

[32]  Cécile Durot.  On the Lp-error of monotonicity constrained estimators , 2007 .

[33]  G. Isac,et al.  Projection methods, isotone projection cones, and the complementarity problem , 1990 .

[34]  R. Tibshirani,et al.  Linear Smoothers and Additive Models , 1989 .

[35]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[36]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .