Learning prototypes and distances (LPD). A prototype reduction technique based on nearest neighbor error minimization

A prototype reduction algorithm is proposed which simultaneous train both a reduced set of prototypes and a suitable local metric for these prototypes. Starting with an initial selection of a small number of prototypes, it iteratively adjusts both the position (features) of these prototypes and the corresponding local-metric weights. The resulting prototypes/metric combination minimizes a suitable estimation of the classification error probability. Good performance of this algorithm is assessed through experiments with a number of benchmark data sets and through a real two-class classification task which consists of detecting human faces in unrestricted-background pictures.

[1]  Shlomo Geva,et al.  Adaptive nearest neighbor pattern classification , 1991, IEEE Trans. Neural Networks.

[2]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[3]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[4]  Claire Cardie,et al.  Examining Locally Varying Weights for Nearest Neighbor Algorithms , 1997, ICCBR.

[5]  Sébastien Marcel,et al.  Face Verification using LDA and MLP on the BANCA database , 2003 .

[6]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification and Regression , 1995, NIPS.

[7]  Anil K. Jain,et al.  Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Enrique Vidal,et al.  Weighting prototypes - a new editing approach , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[9]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[10]  T. Kohonen,et al.  Statistical pattern recognition with neural networks: benchmarking studies , 1988, IEEE 1988 International Conference on Neural Networks.

[11]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Sébastien Marcel,et al.  A symmetric transformation for LDA-based face verification , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[13]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[14]  Takeo Kanade,et al.  Probabilistic modeling of local appearance and spatial relationships for object recognition , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[15]  Francesc J. Ferri,et al.  An efficient prototype merging strategy for the condensed 1-NN rule through class-conditional hierarchical clustering , 2002, Pattern Recognit..

[16]  Enrique Vidal,et al.  An evaluation of the WPE algorithm using tangent distance , 2002, Object recognition supported by user interaction for service robots.

[17]  Ron Kohavi,et al.  The Utility of Feature Weighting in Nearest-Neighbor Algorithms , 1997 .

[18]  Jean-Philippe Thiran,et al.  The BANCA Database and Evaluation Protocol , 2003, AVBPA.

[19]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[20]  Francesco Ricci,et al.  Data Compression and Local Metrics for Nearest Neighbor Classification , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Tony R. Martinez,et al.  Value Difference Metrics for Continuously Valued Attributes , 1996 .

[22]  Dimitrios Gunopulos,et al.  Locally Adaptive Metric Nearest-Neighbor Classification , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Jing Peng,et al.  Adaptive quasiconformal kernel nearest neighbor classification , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Aristidis Likas,et al.  Class Conditional Density Estimation Using Mixtures with Constrained Component Sharing , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Enrique Vidal,et al.  A class-dependent weighted dissimilarity measure for nearest neighbor classification problems , 2000, Pattern Recognit. Lett..

[27]  Roberto Paredes Palacios.  Técnicas para la mejora de la clasificación por el vecino más cercano , 2003 .

[28]  Chin-Liang Chang,et al.  Finding Prototypes For Nearest Neighbor Classifiers , 1974, IEEE Transactions on Computers.

[29]  Jing Peng,et al.  Adaptive kernel metric nearest neighbor classification , 2002, Object recognition supported by user interaction for service robots.

[30]  Takeo Kanade,et al.  A statistical method for 3D object detection applied to faces and cars , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[31]  C. G. Hilborn,et al.  The Condensed Nearest Neighbor Rule , 1967 .