Making metric learning algorithms invariant to transformations using a projection metric on Grassmann manifolds

The requirement for suitable ways to measure the distance or similarity between data is omnipresent in machine learning, pattern recognition and data mining, but extracting such good metrics for particular problems is in general challenging. This has led to the emergence of metric learning ideas, which intend to automatically learn a distance function tuned to a specific task. In many tasks and data types, there are natural transformations to which the classification result should be invariant or insensitive. This demand and its implications are essential in many machine learning applications, and insensitivity to image transformations was in the first place achieved by using invariant feature vectors. In this paper, a new representation model on Grassmann manifolds for data points and a novel method for learning a Mahalanobis metric which uses the geodesic distance on Grassmann manifolds are proposed. In fact, we use an appropriate geodesic distance metric on the Grassmann manifolds, called projection metric, for measuring primary similarities between the new representations of the data points. This makes learning of the Mahalanobis metric invariant to similarity transforms and intensity changes, and therefore improve the performance. Experiments on face and handwritten digit datasets demonstrate that our proposed method yields performance improvements in a state-of-the-art metric learning algorithm.

[1]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[2]  Cordelia Schmid,et al.  Is that you? Metric learning approaches for face identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[3]  George R. Thoma,et al.  RSILC: Rotation- and Scale-Invariant, Line-based Color-aware descriptor , 2015, Image Vis. Comput..

[4]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[5]  Laurent Wendling,et al.  Integrating vocabulary clustering with spatial relations for symbol recognition , 2013, International Journal on Document Analysis and Recognition (IJDAR).

[6]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[7]  Laurent Wendling,et al.  Overlaid Arrow Detection for Labeling Regions of Interest in Biomedical Images , 2016, IEEE Intelligent Systems.

[8]  Daniel D. Lee,et al.  Grassmann discriminant analysis: a unifying view on subspace-based learning , 2008, ICML '08.

[9]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[10]  Alexandros Kalousis,et al.  Parametric Local Metric Learning for Nearest Neighbor Classification , 2012, NIPS.

[11]  Peng Li,et al.  Distance Metric Learning with Eigenvalue Optimization , 2012, J. Mach. Learn. Res..

[12]  Hongdong Li,et al.  Expanding the Family of Grassmannian Kernels: An Embedding Perspective , 2014, ECCV.

[13]  Brian C. Lovell,et al.  Dictionary Learning and Sparse Coding on Grassmann Manifolds: An Extrinsic Solution , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Dong Wang,et al.  Robust Distance Metric Learning via Bayesian Inference , 2018, IEEE Transactions on Image Processing.

[15]  Luiz Eduardo Soares de Oliveira,et al.  Handwritten digit segmentation: Is it still necessary? , 2018, Pattern Recognit..

[16]  Horst Bischof,et al.  Large scale metric learning from equivalence constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  George R. Thoma,et al.  Line Segment-Based Stitched Multipanel Figure Separation for Effective Biomedical CBIR , 2017, Int. J. Pattern Recognit. Artif. Intell..

[18]  Suvrit Sra,et al.  Geometric Mean Metric Learning , 2016, ICML.

[19]  K. C. Santosh Character Recognition Based on DTW-Radon , 2011, 2011 International Conference on Document Analysis and Recognition.

[20]  Xilin Chen,et al.  Projection Metric Learning on Grassmann Manifold with Application to Video based Face Recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Sanjay R. Patil,et al.  Content Based Image Retrieval Using Various Distance Metrics , 2010, ICDEM.

[22]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[23]  Anuj Srivastava,et al.  Bayesian and geometric subspace tracking , 2004, Advances in Applied Probability.

[24]  Wei Liu,et al.  Semi-supervised distance metric learning for collaborative image retrieval and clustering , 2010, ACM Trans. Multim. Comput. Commun. Appl..

[25]  Gert R. G. Lanckriet,et al.  Efficient Learning of Mahalanobis Metrics for Ranking , 2014, ICML.

[26]  David G. Stork,et al.  Pattern Classification , 1973 .

[27]  Bin Fan,et al.  Beyond Mahalanobis metric: Cayley-Klein metric learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  P. Halmos A Hilbert Space Problem Book , 1967 .

[29]  Jing Peng,et al.  Adaptive kernel metric nearest neighbor classification , 2002, Object recognition supported by user interaction for service robots.

[30]  Lei Wang,et al.  Positive Semidefinite Metric Learning Using Boosting-like Algorithms , 2011, J. Mach. Learn. Res..

[31]  Dimitrios Gunopulos,et al.  Adaptive Nearest Neighbor Classification Using Support Vector Machines , 2001, NIPS.

[32]  Feiping Nie,et al.  Learning a Mahalanobis distance metric for data clustering and classification , 2008, Pattern Recognit..

[33]  Partha Pratim Roy,et al.  Arrow detection in biomedical images using sequential classifier , 2018, Int. J. Mach. Learn. Cybern..

[34]  Tomer Hertz,et al.  Learning a Mahalanobis Metric from Equivalence Constraints , 2005, J. Mach. Learn. Res..

[35]  H. Le,et al.  On Geodesics in Euclidean Shape Spaces , 1991 .

[36]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.