Feasibility based Large Margin Nearest Neighbor metric learning

Large margin nearest neighbor (LMNN) is a metric learner which optimizes the performance of the popular $k$NN classifier. However, its resulting metric relies on pre-selected target neighbors. In this paper, we address the feasibility of LMNN's optimization constraints regarding these target points, and introduce a mathematical measure to evaluate the size of the feasible region of the optimization problem. We enhance the optimization framework of LMNN by a weighting scheme which prefers data triplets which yield a larger feasible region. This increases the chances to obtain a good metric as the solution of LMNN's problem. We evaluate the performance of the resulting feasibility-based LMNN algorithm using synthetic and real datasets. The empirical results show an improved accuracy for different types of datasets in comparison to regular LMNN.

[1]  Peng Li,et al.  Distance Metric Learning with Eigenvalue Optimization , 2012, J. Mach. Learn. Res..

[2]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[3]  Chunhua Shen,et al.  Efficiently Learning a Distance Metric for Large Margin Nearest Neighbor Classification , 2011, AAAI.

[4]  Qingling Zhang,et al.  Eigenvalue Inequalities for Matrix Product , 2006, IEEE Transactions on Automatic Control.

[5]  Te-son Kuo,et al.  Trace bounds on the solution of the algebraic matrix Riccati and Lyapunov equation , 1986 .

[6]  Narendra Ahuja,et al.  Pedestrian Recognition with a Learned Metric , 2010, ACCV.

[7]  Rama Chellappa,et al.  Hierarchical Multimodal Metric Learning for Multimodal Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Udo Seiffert,et al.  Classification in high-dimensional spectral data: Accuracy vs. interpretability vs. model size , 2014, Neurocomputing.

[9]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[10]  Kilian Q. Weinberger,et al.  Large Margin Multi-Task Metric Learning , 2010, NIPS.

[11]  Brian Kulis,et al.  Metric Learning: A Survey , 2013, Found. Trends Mach. Learn..

[12]  Jie Yang,et al.  Adaptive Multi-Metric Fusion for Person Re-identification , 2016, CCPR.

[13]  Barbara Hammer,et al.  Efficient metric learning for the analysis of motion data , 2015, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[14]  Marc Sebban,et al.  A Survey on Metric Learning for Feature Vectors and Structured Data , 2013, ArXiv.

[15]  Michael Biehl,et al.  Analysis of Flow Cytometry Data by Matrix Relevance Learning Vector Quantization , 2013, PloS one.

[16]  Barbara Hammer,et al.  Convergence of Multi-pass Large Margin Nearest Neighbor Metric Learning , 2016, ICANN.

[17]  Shiliang Sun,et al.  Hierarchical Large Margin Nearest Neighbor Classification , 2010, 2010 20th International Conference on Pattern Recognition.

[18]  Cordelia Schmid,et al.  Is that you? Metric learning approaches for face identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[19]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.