Metric learning by discriminant neighborhood embedding

In this paper, we learn a distance metric in favor of classification task from available labeled samples. Multi-class data points are supposed to be pulled or pushed by discriminant neighbors. We define a discriminant adjacent matrix in favor of classification task and learn a map transforming input data into a new space such that intra-class neighbors become even more nearby while extra-class neighbors become as far away from each other as possible. Our method is non-parametric, non-iterative, and immune to small sample size (SSS) problem. Target dimensionality of the new space is selected by spectral analysis in the proposed method. Experiments on real-world data sets demonstrate the effectiveness of our method.

[1]  Dimitrios Gunopulos,et al.  Locally Adaptive Metric Nearest-Neighbor Classification , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Zhihua Zhang,et al.  Parametric Distance Metric Learning with Label Information , 2003, IJCAI.

[3]  Xiangyang Xue,et al.  Discriminant neighborhood embedding for classification , 2006, Pattern Recognit..

[4]  Hwann-Tzong Chen,et al.  Local discriminant embedding and its variants , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[6]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[7]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[8]  Lide Wu,et al.  Stepwise Nearest Neighbor Discriminant Analysis , 2005, IJCAI.

[9]  Yan Qiu Chen,et al.  Improving nearest neighbor classification with cam weighted distance , 2006, Pattern Recognit..

[10]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[12]  Yoram Singer,et al.  Online and batch learning of pseudo-metrics , 2004, ICML.

[13]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[14]  Tomer Hertz,et al.  Learning Distance Functions using Equivalence Relations , 2003, ICML.

[15]  Dimitrios Gunopulos,et al.  Large margin nearest neighbor classifiers , 2005, IEEE Transactions on Neural Networks.

[16]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[17]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[18]  Robert P. W. Duin,et al.  Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Slobodan Ribarić,et al.  Introduction to Pattern Recognition , 1988 .

[20]  GunopulosDimitrios,et al.  Locally Adaptive Metric Nearest-Neighbor Classification , 2002 .

[21]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[22]  M. Bressan,et al.  Nonparametric discriminant analysis and nearest neighbor classification , 2003, Pattern Recognit. Lett..

[23]  K. Fukunaga,et al.  Nonparametric Discriminant Analysis , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[25]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[26]  Shuicheng Yan,et al.  Graph embedding: a general framework for dimensionality reduction , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).