An Optimal Global Nearest Neighbor Metric

A quadratic metric dAO (X, Y) =[(X - Y)T AO(X - Y)]¿ is proposed which minimizes the mean-squared error between the nearest neighbor asymptotic risk and the finite sample risk. Under linearity assumptions, a heuristic argument is given which indicates that this metric produces lower mean-squared error than the Euclidean metric. A nonparametric estimate of Ao is developed. If samples appear to come from a Gaussian mixture, an alternative, parametrically directed distance measure is suggested for nearness decisions within a limited region of space. Examples of some two-class Gaussian mixture distributions are included.

[1]  D. Fraser Nonparametric methods in statistics , 1957 .

[2]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[3]  Larry D. Hostetler,et al.  Optimization of k nearest neighbor density estimates , 1973, IEEE Trans. Inf. Theory.

[4]  Keinosuke Fukunaga,et al.  Nonparametric Bayes error estimation using unclassified samples , 1972, IEEE Trans. Inf. Theory.

[5]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[6]  Godfried T. Toussaint,et al.  Bibliography on estimation of misclassification , 1974, IEEE Trans. Inf. Theory.

[7]  Larry D. Hostetler,et al.  k-nearest-neighbor Bayes-risk estimation , 1975, IEEE Trans. Inf. Theory.

[8]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[9]  Stephen S. Yau,et al.  Nonparametric Estimation of the Bayes Error of Feature Extractors Using Ordered Nearest Neighbor Sets , 1977, IEEE Transactions on Computers.

[10]  Jack Koplowitz,et al.  The weighted nearest neighbor rule for class dependent sample sizes (Corresp.) , 1979, IEEE Trans. Inf. Theory.

[11]  Pierre A. Devijver,et al.  New error bounds with the nearest neighbor rule (Corresp.) , 1979, IEEE Trans. Inf. Theory.

[12]  Keinosuke Fukunaga,et al.  The optimal distance measure for nearest neighbor classification , 1981, IEEE Trans. Inf. Theory.

[13]  K. Fukunaga,et al.  Nonparametric Discriminant Analysis , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.