An affine invariant k-nearest neighbor regression estimate

We propose a new k-NN regression estimate based on a data-dependent metric in Rd which is used to define the k-nearest neighbors of a given point. The metric is invariant under all affine transformations. With this metric, the standard k-nearest neighbor regression estimate is asymptotically consistent under the usual conditions on k, and minimal requirements on the input data.

[1]  R. Randles.  A Distribution-Free Multivariate Sign Test Based on Interdirections , 1989 .

[2]  Bernard Chazelle,et al.  Cutting hyperplanes for divide-and-conquer , 1993, Discret. Comput. Geom..

[3]  T. Broadbent.  Measure and Integral , 1957, Nature.

[4]  A. Zygmund,et al.  Measure and integral : an introduction to real analysis , 1977 .

[5]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[6]  Luc Devroye,et al.  On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification , 2010, J. Multivar. Anal..

[7]  Hannu Oja,et al.  The geometry of the affine invariant multivariate sign and rank methods , 1999 .

[8]  Adam Krzyzak,et al.  New Multivariate Product Density Estimators , 2002 .

[9]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[10]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[11]  J. Marcinkiewicz.  Sur les fonctions indépendantes , 1938 .

[12]  Hannu Oja,et al.  OPERATING TRANSFORMATION RETRANSFORMATION ON SPATIAL MEDIAN AND ANGLE TEST , 1998 .

[13]  S. Meiser,et al.  Point Location in Arrangements of Hyperplanes , 1993, Inf. Comput..

[14]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[15]  D. Paindaveine,et al.  Optimal tests for multivariate location based on interdirections and pseudo-Mahalanobis ranks , 2002 .

[16]  Hannu Oja,et al.  Affine Invariant Multivariate Sign and Rank Tests and Corresponding Estimates: a Review , 1999 .

[17]  David E. Tyler.  A Distribution-Free $M$-Estimator of Multivariate Scatter , 1987 .

[18]  M. Gessaman,et al.  A Comparison of Some Multivariate Discrimination Procedures , 1972 .

[19]  M. Samanta,et al.  A note on uniform strong convergence of bivariate density estimates , 1974 .

[20]  R. Randles,et al.  A practical affine equivariant multivariate median , 2002 .

[21]  Hannu Oja,et al.  Estimates of Regression Coefficients Based on Lift Rank Covariance Matrix , 2003 .

[22]  M. Gessaman.  A Consistent Nonparametric Multivariate Density Estimator Based on Statistically Equivalent Blocks , 1970 .

[23]  H. O. Posten.  Multidimensional Gaussian Distributions , 1964 .

[24]  Hannu Oja,et al.  Estimates of regression coefficients based on the sign covariance matrix , 2002 .

[25]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[26]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[27]  A. Shiryayev.  On Sums of Independent Random Variables , 1992 .

[28]  Thomas M. Cover,et al.  Estimation by the nearest neighbor rule , 1968, IEEE Trans. Inf. Theory.

[29]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[30]  O. Gaans.  Probability measures on metric spaces , 2022 .

[31]  R. Olshen,et al.  Asymptotically Efficient Solutions to the Classification Problem , 1978 .

[32]  R. Olshen,et al.  Consistent nonparametric regression from recursive partitioning schemes , 1980 .

[33]  M. Gessaman,et al.  Nonparametric Discrimination Using Tolerance Regions , 1968 .

[34]  Hannu Oja,et al.  Optimal signed-rank tests based on hyperplanes , 2005 .