Nearest neighbor search by using Partial KD-tree method

We present a new nearest neighbor (NN) search algorithm, the Partial KD-Tree Search (PKD), which couples the Friedman’s algorithm and the Partial Distance Search (PDS ) technique. Its efficiency was tested using a wide spectrum of input datasets of various sizes and dimensions. The test datasets were both generated artificially and selected from the UCI repository. It appears that our hybrid algorithm is very efficient in comparison to its components and to other popular NN search technique – the Slicing Search algorithm. The results of tests show that PKD outperforms up to 100 times the brute force method and is substantially faster than other techniques. We can conclude that the Partial KD-Tree is a universal and effcient nearest neighbor search scheme.

[1]  David G. Stork,et al.  Pattern Classification , 1973 .

[2]  Forest Baskett,et al.  An Algorithm for Finding Nearest Neighbors , 1975, IEEE Transactions on Computers.

[3]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[4]  Allen Gersho,et al.  Fast search algorithms for vector quantization and pattern matching , 1984, ICASSP.

[5]  J. Bentley.  A survey of techniques for fixed radius near neighbor searching. , 1975 .

[6]  Belur V. Dasarathy,et al.  Minimal consistent set (MCS) identification for optimal nearest neighbor decision systems design , 1994, IEEE Trans. Syst. Man Cybern..

[7]  Kazuo Hattori,et al.  A new edited k-nearest neighbor rule in the pattern classification problem , 2000, Pattern Recognit..

[8]  Ludmila I. Kuncheva,et al.  Fitness functions in editing k-NN reference set by genetic algorithms , 1997, Pattern Recognit..

[9]  Shree K. Nayar,et al.  Closest point search in high dimensions , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  S. Arya.  Nearest neighbor searching and applications , 1996 .

[11]  Luisa Micó,et al.  A fast branch & bound nearest neighbour classifier in metric spaces , 1996, Pattern Recognit. Lett..

[12]  Belur V. Dasarathy,et al.  Nearest neighbor (NN) norms: NN pattern classification techniques , 1991 .

[13]  Dennis L. Wilson,et al.  Asymptotic Properties of Nearest Neighbor Rules Using Edited Data , 1972, IEEE Trans. Syst. Man Cybern..

[14]  Donald E. Knuth,et al.  The Art of Computer Programming: Volume 3: Sorting and Searching , 1998 .

[15]  Hong Yan,et al.  Prototype optimization for nearest neighbor classifiers using a two-layer perceptron , 1993, Pattern Recognit..

[16]  Peter N. Yianilos,et al.  Data structures and algorithms for nearest neighbor search in general metric spaces , 1993, SODA '93.

[17]  Thomas P. Yunck,et al.  A Technique to Identify Nearest Neighbors , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[18]  W. Eric L. Grimson,et al.  Prototype optimization for nearest-neighbor classification , 2002, Pattern Recognit..

[19]  C.-C. Jay Kuo,et al.  Fast tree-structured nearest neighbor encoding for vector quantization , 1996, IEEE Trans. Image Process..

[20]  Sameer A. Nene,et al.  A simple algorithm for nearest neighbor search in high dimensions , 1997 .

[21]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[22]  Donald E. Knuth,et al.  The Art of Computer Programming, Volume I: Fundamental Algorithms, 2nd Edition , 1997 .

[23]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[24]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Small Sample Performance , 1952 .

[25]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[26]  Song B. Park,et al.  A Fast k Nearest Neighbor Finding Algorithm Based on the Ordered Partition , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Computing k-Nearest Neighbors , 1975, IEEE Transactions on Computers.

[28]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.