Metric learning with feature decomposition for image categorization

This paper proposes an approach named distance metric learning with feature decomposition (DMLFD) that can reduce the computational costs of distance metric learning (DML) methods as well as improve their performance for image categorization. We have analyzed that a high-dimensional feature space with limited training data will introduce difficulties to DML algorithms in both computation and performance. To tackle these difficulties, we decompose the high-dimensional feature space into a set of low-dimensional feature spaces with minimal dependencies. Then we perform DML for each low-dimensional feature space to construct a sub-metric and the sub-metrics are finally combined into a global metric. We conduct experiments on Corel and TRECVID datasets with different DML methods, and empirical results have demonstrated the effectiveness and efficiency of the proposed approach.

[1]  Amir Globerson,et al.  Metric Learning by Collapsing Classes , 2005, NIPS.

[2]  Changhu Wang,et al.  Learning to reduce the semantic gap in web image retrieval and annotation , 2008, SIGIR '08.

[3]  Yi Liu,et al.  An Efficient Algorithm for Local Distance Metric Learning , 2006, AAAI.

[4]  Jieping Ye,et al.  An optimization criterion for generalized discriminant analysis on undersampled problems , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Xuelong Li,et al.  Modality Mixture Projections for Semantic Video Event Detection , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[7]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[8]  Hong Chang,et al.  Extending the relevant component analysis algorithm for metric learning using both positive and negative equivalence constraints , 2006, Pattern Recognit..

[9]  Wei-Ying Ma,et al.  Graph based multi-modality learning , 2005, ACM Multimedia.

[10]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[11]  Edward Y. Chang,et al.  Optimal multimodal fusion for multimedia data analysis , 2004, MULTIMEDIA '04.

[12]  Meng Wang,et al.  Unified Video Annotation via Multigraph Learning , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  W. V. McCarthy,et al.  Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data , 1995 .

[15]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[16]  Xuelong Li,et al.  General Averaged Divergence Analysis , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[17]  Tomer Hertz,et al.  Learning Distance Functions using Equivalence Relations , 2003, ICML.

[18]  Meng Wang,et al.  Study on the combination of video concept detectors , 2008, ACM Multimedia.

[19]  Edward Y. Chang,et al.  Manifold learning, a promised land or work in progress? , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[20]  Wei Liu,et al.  Semi-supervised distance metric learning for Collaborative Image Retrieval , 2008, CVPR.

[21]  Wei Liu,et al.  Learning Distance Metrics with Contextual Constraints for Image Retrieval , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[22]  Wei Liu,et al.  Output Regularized Metric Learning with Side Information , 2008, ECCV.

[23]  Ivor W. Tsang,et al.  Learning with Idealized Kernels , 2003, ICML.

[24]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Jingrui He,et al.  Manifold-ranking based image retrieval , 2004, MULTIMEDIA '04.

[26]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[27]  Kilian Q. Weinberger,et al.  Metric Learning for Kernel Regression , 2007, AISTATS.

[28]  Jonathan Goldstein,et al.  When Is ''Nearest Neighbor'' Meaningful? , 1999, ICDT.

[29]  Xian-Sheng Hua,et al.  A joint appearance-spatial distance for kernel-based image categorization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[31]  Xuelong Li,et al.  Tensor Rank One Discriminant Analysis - A convergent method for discriminative multilinear subspace selection , 2008, Neurocomputing.

[32]  Moni Naor,et al.  Optimal aggregation algorithms for middleware , 2001, PODS '01.

[33]  Meng Wang,et al.  Beyond Distance Measurement: Constructing Neighborhood Similarity for Video Annotation , 2009, IEEE Transactions on Multimedia.

[34]  Shih-Fu Chang,et al.  Columbia University’s Baseline Detectors for 374 LSCOM Semantic Visual Concepts , 2007 .

[35]  Mahdieh Soleymani Baghshah,et al.  Semi-Supervised Metric Learning Using Pairwise Constraints , 2009, IJCAI.