A review on multi-task metric learning

Distance metric plays an important role in machine learning which is crucial to the performance of a range of algorithms. Metric learning, which refers to learning a proper distance metric for a particular task, has attracted much attention in machine learning. In particular, multi-task learning deals with the scenario where there are multiple related metric learning tasks. By jointly training these tasks, useful information is shared among the tasks, which significantly improves their performances. This paper reviews the literature on multi-task metric learning. Various methods are investigated systematically and categorized into four families. The central ideas of these methods are introduced in detail, followed by some representative applications. Finally, we conclude the review and propose a number of future work directions.

[1]  Kilian Q. Weinberger,et al.  Large Margin Multi-Task Metric Learning , 2010, NIPS.

[2]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[3]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[4]  André Elisseeff,et al.  Stability and Generalization , 2002, J. Mach. Learn. Res..

[5]  Amaury Habrard,et al.  Robustness and generalization for metric learning , 2012, Neurocomputing.

[6]  Ricardo Vilalta,et al.  A Perspective View and Survey of Meta-Learning , 2002, Artificial Intelligence Review.

[7]  Zenglin Xu,et al.  Robust Metric Learning by Smooth Optimization , 2010, UAI.

[8]  Liwei Wang,et al.  Theory and Algorithm for Learning with Dissimilarity Functions , 2009, Neural Computation.

[9]  Rong Jin,et al.  Regularized Distance Metric Learning: Theory and Algorithm , 2009, NIPS.

[10]  Jonathan Baxter,et al.  A Model of Inductive Bias Learning , 2000, J. Artif. Intell. Res..

[11]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[12]  Babak Nadjar Araabi,et al.  Deep Multitask Metric Learning for Offline Signature Verification , 2016, Pattern Recognit. Lett..

[13]  A. Rukhin.  Matrix Variate Distributions , 1999, The Multivariate Normal Distribution.

[14]  Dit-Yan Yeung,et al.  Transfer Metric Learning with Semi-Supervised Extension , 2012, TIST.

[15]  Marc Sebban,et al.  A Survey on Metric Learning for Feature Vectors and Structured Data , 2013, ArXiv.

[16]  Dacheng Tao,et al.  Person Re-Identification Over Camera Networks Using Multi-Task Distance Metric Learning , 2014, IEEE Transactions on Image Processing.

[17]  Yuan Shi,et al.  Sparse Compositional Metric Learning , 2014, AAAI.

[18]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[19]  Lorenzo Torresani,et al.  Large Margin Component Analysis , 2006, NIPS.

[20]  Dit-Yan Yeung,et al.  Transfer metric learning by learning task relationships , 2010, KDD.

[21]  Kaizhu Huang,et al.  Sparse Metric Learning via Smooth Optimization , 2009, NIPS.

[22]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[23]  Hong Liu,et al.  Two-level multi-task metric learning with application to multi-classification , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[24]  Yiming Yang,et al.  Flexible latent variable models for multi-task learning , 2008, Machine Learning.

[25]  Sebastian Thrun,et al.  Learning to Learn , 1998, Springer US.

[26]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[27]  Jonathan Baxter,et al.  A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling , 1997, Machine Learning.

[28]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[29]  Jiwen Lu,et al.  Discriminative Deep Metric Learning for Face Verification in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Dit-Yan Yeung,et al.  A Convex Formulation for Learning Task Relationships in Multi-Task Learning , 2010, UAI.

[31]  Geoffrey E. Hinton,et al.  Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure , 2007, AISTATS.

[32]  Ivor W. Tsang,et al.  Learning with Idealized Kernels , 2003, ICML.

[33]  Horst Bischof,et al.  Large scale metric learning from equivalence constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Maria-Florina Balcan,et al.  A theory of learning with similarity functions , 2008, Machine Learning.

[35]  Jianping Fan,et al.  Hierarchical learning of multi-task sparse metrics for large-scale image classification , 2017, Pattern Recognit..

[36]  Jesús Martínez del Rincón,et al.  Person Reidentification Using Deep Convnets With Multitask Learning , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Rama Chellappa,et al.  Domain adaptation for object recognition: An unsupervised approach , 2011, 2011 International Conference on Computer Vision.

[38]  Amaury Habrard,et al.  A Theoretical Analysis of Metric Hypothesis Transfer Learning , 2015, ICML.

[39]  Kaizhu Huang,et al.  A multi-task framework for metric learning with common subspace , 2012, Neural Computing and Applications.

[40]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[41]  Peng Li,et al.  Distance Metric Learning with Eigenvalue Optimization , 2012, J. Mach. Learn. Res..

[42]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[43]  Inderjit S. Dhillon,et al.  Matrix Nearness Problems with Bregman Divergences , 2007, SIAM J. Matrix Anal. Appl..

[44]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[45]  Inderjit S. Dhillon,et al.  Low-Rank Kernel Learning with Bregman Matrix Divergences , 2009, J. Mach. Learn. Res..

[46]  Gaurav Sharma,et al.  CP-mtML: Coupled Projection Multi-Task Metric Learning for Large Scale Face Retrieval , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Kaizhu Huang,et al.  Generalized sparse metric learning with relative comparisons , 2011, Knowledge and Information Systems.

[48]  Qi Tian,et al.  Multi-feature metric learning with knowledge transfer among semantics and social tagging , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[50]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[51]  Brian Kulis,et al.  Metric Learning: A Survey , 2013, Found. Trends Mach. Learn..

[52]  Dacheng Tao,et al.  Online Semi-Supervised Multi-task Distance Metric Learning , 2016, 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW).

[53]  Koby Crammer,et al.  Learning Bounds for Domain Adaptation , 2007, NIPS.

[54]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[55]  Kaizhu Huang,et al.  Geometry preserving multi-task metric learning , 2012, Machine Learning.

[56]  Charles A. Micchelli,et al.  A Spectral Regularization Framework for Multi-Task Structure Learning , 2007, NIPS.

[57]  D. Burago,et al.  A Course in Metric Geometry , 2001 .

[58]  Sebastian Thrun,et al.  Lifelong Learning Algorithms , 1998, Learning to Learn.

[59]  Kaizhu Huang,et al.  GSML: A Unified Framework for Sparse Metric Learning , 2009, 2009 Ninth IEEE International Conference on Data Mining.