Ranking Fisher discriminant analysis

Abstract Existing feature dimensionality reduction algorithms are inherently designed for the case of classification, clustering and retrieval, but not for ranking applications such as visual search reranking. This is a serious limitation which restricts the applicability of existing dimensionality reduction methods as well as the generalization ability of ranking applications. Therefore, it is important to design a kind of special methods to be effectively employed for ranking applications. Fisher discriminant analysis (FDA) is one of the most popular dimensionality reduction methods. Thus, we propose a novel dimensionality reduction algorithm based on FDA to solve this kind of problem in this paper. Specifically, relevance degree information—the data label in ranking applications, is introduced to a semi-supervised form of FDA, in which both local information and unlabeled data are employed. We name the proposed method as ranking Fisher discriminant analysis (RFDA). To verify the effectiveness of RFDA, extensive experiments are carried out on image search reranking applications, which show significant performance based on the popular MSRA-MM dataset.

[1]  Jiawei Han,et al.  Learning a Maximum Margin Subspace for Image Retrieval , 2008, IEEE Transactions on Knowledge and Data Engineering.

[2]  Dacheng Tao,et al.  Biased Discriminant Euclidean Embedding for Content-Based Image Retrieval , 2010, IEEE Transactions on Image Processing.

[3]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[4]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[5]  Chong-Wah Ngo,et al.  Co-reranking by mutual reinforcement for image search , 2010, CIVR '10.

[6]  Wen Gao,et al.  Maximal Linear Embedding for Dimensionality Reduction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Xuelong Li,et al.  Binary Two-Dimensional PCA , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  Nenghai Yu,et al.  Neighborhood Preserving Projections (NPP): A Novel Linear Dimension Reduction Method , 2005, ICIC.

[9]  Meng Wang,et al.  Beyond Distance Measurement: Constructing Neighborhood Similarity for Video Annotation , 2009, IEEE Transactions on Multimedia.

[10]  John R. Smith,et al.  Data Modeling Strategies for Imbalanced Learning in Visual Search , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[11]  Xian-Sheng Hua,et al.  MSRA-MM: Bridging Research and Industrial Societies for Multimedia Information Retrieval , 2009 .

[12]  Daoqiang Zhang,et al.  Semi-Supervised Dimensionality Reduction ∗ , 2007 .

[13]  Boonserm Kijsirikul,et al.  A unified semi-supervised dimensionality reduction framework for manifold learning , 2008, Neurocomputing.

[14]  Stan Z. Li,et al.  Extraction of feature subspaces for content-based retrieval using relevance feedback , 2001, MULTIMEDIA '01.

[15]  Juyang Weng,et al.  Using Discriminant Eigenfeatures for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Rong Yan,et al.  Multimedia Search with Pseudo-relevance Feedback , 2003, CIVR.

[17]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[19]  Shinichi Nakajima,et al.  Semi-supervised local Fisher discriminant analysis for dimensionality reduction , 2009, Machine Learning.

[20]  Yi-Hsuan Yang,et al.  Online Reranking via Ordinal Informative Concepts for Context Fusion in Concept Detection and Video Search , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Tao Mei,et al.  Optimizing Visual Search Reranking via Pairwise Learning , 2011, IEEE Transactions on Multimedia.

[22]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[23]  Xian-Sheng Hua,et al.  Active Reranking for Web Image Search , 2010, IEEE Transactions on Image Processing.

[24]  Kun Zhou,et al.  Locality Sensitive Discriminant Analysis , 2007, IJCAI.

[25]  Xian-Sheng Hua,et al.  Towards a Relevant and Diverse Search of Social Images , 2010, IEEE Transactions on Multimedia.

[26]  Xiaogang Wang,et al.  Query-specific visual semantic spaces for web image re-ranking , 2011, CVPR 2011.

[27]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[28]  Yao Zhao,et al.  Multimodal Fusion for Video Search Reranking , 2010, IEEE Transactions on Knowledge and Data Engineering.

[29]  Yan Liu,et al.  Semi-supervised manifold ordinal regression for image ranking , 2011, MM '11.

[30]  Yuan Yuan,et al.  Outlier-resisting graph embedding , 2010, Neurocomputing.

[31]  Ralf Herbrich,et al.  Large margin rank boundaries for ordinal regression , 2000 .

[32]  J. Friedman Regularized Discriminant Analysis , 1989 .

[33]  Tanji Hu,et al.  Summarizing tourist destinations by mining user-generated travelogues and photos , 2011, Comput. Vis. Image Underst..

[34]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[35]  David G. Stork,et al.  Pattern Classification , 1973 .

[36]  Robert P. W. Duin,et al.  Expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix , 1998, Pattern Recognit. Lett..

[37]  Zhihua Zhang,et al.  Regularized Discriminant Analysis, Ridge Regression and Beyond , 2010, J. Mach. Learn. Res..