Nonparametric Conditional Estimation

Abstract : Many nonparametric regression techniques (such as kernels, nearest neighbors, and smoothing splines) estimate the conditional mean of Y given X = x by a weighted sum of observed Y values, where observations with X values near x tend to have larger weights. In this report the weights are taken to represent a finite signed measure on the space of Y values. This measure is studied as an estimate of the conditional distribution of Y given X= x. From estimates of the conditional distribution, estimates of conditional means, standard deviations, quantiles and other statistical functionals may be computed. Chapter 1 illustrates the computation of conditional quantiles and conditional survival probabilities on the Stanford Heart Transplant data. Chapter 2 contains a survey of nonparametric regression methods and introduces statistical metrics and von Mises' method for later use. Chapter 3 proves some consistency results. The estimated conditional distribution of Y is shown to be consistent in the following sense: the Prohorov distance between the estimated and true conditional distributions converges in probability to zero. The required conditions are: that the distribution of Y given X = x vary continuously with x, that the weights regarded as a measure on the X space converge in probability to a point mass at x, and that a measure of the effective local sample size tend to infinity in probability. A slight strengthening of the conditions allows one to establish almost sure consistency. Consistency of Prohorov-continuous (i.e. robust) functionals follows immediately. In the above, the X and Y spaces are complete separable metric spaces. In case Y is the real line, weak and strong consistency results are established for the Kolmogorov-Smirnov and the Vasserstein metrics under stronger conditions. Chapter 4 provides conditions under which the suitably normalized errors in estimating the conditional distribution of Y have a Brownian limit.

[1]  D. Freedman,et al.  Some Asymptotic Theory for the Bootstrap , 1981 .

[2]  D. Titterington.  Common structure of smoothing techniques in statistics , 1985 .

[3]  F. Hampel.  A General Qualitative Definition of Robustness , 1971 .

[4]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[5]  B. Efron.  The two sample problem with censored data , 1967 .

[6]  M. Segal.  Recursive Partitioning Using Ranks. , 1985 .

[7]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[8]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[9]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[10]  J. K. Benedetti.  On the Nonparametric Estimation of Regression Functions , 1977 .

[11]  Jerome H. Friedman,et al.  Smoothing of Scatterplots , 1982 .

[12]  Pi-Erh Lin,et al.  Nonparametric estimation of a regression function , 1981 .

[13]  G. S. Watson.  Smoothing and interpolation by kriging and with splines , 1984 .

[14]  Peter Hall,et al.  Asymptotic properties of integrated square error and cross-validation for kernel estimation of a regression function , 1984 .

[15]  John Alan McDonald,et al.  Smoothing with split linear fits , 1986 .

[16]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[17]  W. Cleveland.  Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[18]  N. Hjort,et al.  Bayesian Nonparametric Bootstrap Confidence Intervals , 1985 .

[19]  P. Major.  On the invariance principle for sums of independent identically distributed random variables , 1978 .

[20]  M. Braga,et al.  Exploratory Data Analysis , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[21]  C. Reinsch.  Smoothing by spline functions , 1967 .

[22]  L. Fernholz.  von Mises Calculus For Statistical Functionals , 1983 .

[23]  Robert Tibshirani,et al.  The Bootstrap Method for Assessing Statistical Accuracy , 1985 .

[24]  T. Cipra.  Statistical Analysis of Time Series , 2010 .

[25]  W. Stute.  Asymptotic Normality of Nearest Neighbor Regression Function Estimates , 1984 .

[26]  J. Norris.  Appendix: probability and measure , 1997 .

[27]  B. Silverman,et al.  Weak and strong uniform consistency of kernel regression estimates , 1982 .

[28]  Jan Pedersen,et al.  Computing environments for data analysis. Part 2. Hardware , 1985 .

[29]  C. Mallows.  A Note on Asymptotic Joint Normality , 1972 .

[30]  B. Silverman,et al.  Spline Smoothing: The Equivalent Variable Kernel Method , 1984 .

[31]  Naomi S. Altman.  Expert Systems and Statistical Expertise. Part 1. Statistical Expert Systems , 1985 .

[32]  J. Friedman.  A VARIABLE SPAN SMOOTHER , 1984 .

[33]  IItevor Hattie.  Principal Curves and Surfaces , 1984 .

[34]  Jerome H. Friedman,et al.  SMART User's Guide , 1984 .

[35]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[36]  I. W. Wright.  Splines in Statistics , 1983 .

[37]  Z. Bai,et al.  STRONG CONSISTENCY OF THE NEARESTNEIGHBOR ESTIMATES OF NONPARAMETRIC REGRESSION FUNCTIONS , 1984 .

[38]  H. Bergström,et al.  Weak convergence of measures , 1982 .

[39]  Nils L. Hjort.  On Frequency Polygons and Average Shifted Histograms in Higher Dimensions , 1986 .

[40]  C. Quesenberry,et al.  A nonparametric estimate of a multivariate density function , 1965 .

[41]  V. A. Epanechnikov.  Non-Parametric Estimation of a Multivariate Probability Density , 1969 .

[42]  H. D. Brunk,et al.  Discussion: Consistent Nonparametric Regression , 1977 .

[43]  N. Hjort.  Bootstrapping Cox's Regression Model. , 1985 .

[44]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[45]  G. Wahba,et al.  A completely automatic french curve: fitting spline functions by cross validation , 1975 .

[46]  B. Efron.  Better Bootstrap Confidence Intervals , 1987 .

[47]  Winfried Stute,et al.  Conditional empirical processes , 1986 .

[48]  D. W. Scott,et al.  Biased and Unbiased Cross-Validation in Density Estimation , 1987 .

[49]  M. Johns.  Fully nonparametric empirical Bayes estimation via projection pursuit , 1986 .

[50]  J. McDonald.  Computing Environments for Data Analysis Part 3 : Programming Environments ∗ , 1987 .

[51]  E. Nadaraya.  On Non-Parametric Estimates of Density Functions and Regression Curves , 1965 .

[52]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[53]  R. Bass,et al.  Review: P. Billingsley, Convergence of probability measures , 1971 .

[54]  L. Devroye.  On the Almost Everywhere Convergence of Nonparametric Regression Function Estimates , 1981 .

[55]  A. Owen,et al.  Consistency of Smoothing with Running Linear Fits , 1984 .

[56]  Kai Lai Chung,et al.  A Course in Probability Theory , 1949 .

[57]  L. Devroye.  Necessary and sufficient conditions for the pointwise convergence of nearest neighbor regression function estimates , 1982 .

[58]  Shie-Shien Yang,et al.  Linear Functions of Concomitants of Order Statistics with Application to Nonparametric Estimation of a Regression Function , 1981 .

[59]  R. Tibshirani,et al.  Local Likelihood Estimation , 1987 .

[60]  J. Marsden,et al.  Lectures on analysis , 1969 .

[61]  S. Yakowitz,et al.  A comparison of kriging with nonparametric regression methods , 1985 .

[62]  William F. Stout.  Some Results on the Complete and Almost Sure Convergence of Linear Combinations of Independent Random Variables and Martingale Differences , 1968 .

[63]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[64]  Joseph C Marhoul,et al.  A Model for Large Sparse Contingency Tables , 1984 .

[65]  R. Dobrushin.  Prescribing a System of Random Variables by Conditional Distributions , 1970 .

[66]  Zhe Jiang,et al.  Spatial Statistics , 2013 .

[67]  J. Lamperti.  ON CONVERGENCE OF STOCHASTIC PROCESSES , 1962 .

[68]  H. Müller,et al.  Kernel estimation of regression functions , 1979 .

[69]  A. Owen.  Nonparametric Likelihood Ratio Intervals , 1985 .