Supervised distance metric learning through maximization of the Jeffrey divergence

Over the past decades, distance metric learning has attracted a lot of interest in machine learning and related fields. In this work, we propose an optimization framework for distance metric learning via linear transformations by maximizing the Jeffrey divergence between two multivariate Gaussian distributions derived from local pairwise constraints. In our method, the distance metric is trained on positive and negative difference spaces, which are built from the neighborhood of each training instance, so that the local discriminative information is preserved. We show how to solve this problem with a closed-form solution rather than using tedious optimization procedures. The solution is easy to implement, and tractable for large-scale problems. Experimental results are presented for both a linear and a kernelized version of the proposed method for k-nearest neighbors classification. We obtain classification accuracies superior to the state-of-the-art distance metric learning methods in several cases while being competitive in others. HighlightsWe propose a novel distance metric learning method (DMLMJ) for classification.DMLMJ is simple to implement and it can be solved analytically.We extend DMLMJ into a kernelized version to tackle non-linear problems.Experiments on several data sets show the effectiveness of the proposed method.