On integrating re-ranking and rank list fusion techniques for image retrieval

This paper aims to unify image re-ranking and rank aggregation strategies to enhance the retrieval precision of content-based image retrieval (CBIR) systems. In general, CBIR systems are concerned with the retrieval of a set of relevant images from large repositories in response to a submitted query. The primary objective of CBIR systems is the exact ordering of database images in accordance with the presented query. To this end, we present a novel image re-ranking scheme for reordering the initial search results returned by multiple retrieval models and an efficient rank list fusion scheme to combine these refined retrieval results to achieve better performance. The re-ranking algorithm introduced in this work utilizes distance correlation coefficient to refine the search result generated by a given retrieval model. It involves two-step clustering of the initial retrieval list followed by an adaptive procedure for updating the similarity scores among images based on the created clusters. Similarly, the Particle Swarm Optimization-based similarity score fusion framework presented in this work optimally combines the retrieval results generated by multiple CBIR systems. The proposed approach is evaluated on various retrieval tasks using state-of-the-art low-level and high-level descriptors. Experimental results show that our model can significantly enhance the overall effectiveness of CBIR systems.

[1]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[2]  Longin Jan Latecki,et al.  Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval , 2009, CVPR.

[3]  Zhuowen Tu,et al.  Improving Shape Retrieval by Learning Graph Transduction , 2008, ECCV.

[4]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Charles L. A. Clarke,et al.  Reciprocal rank fusion outperforms condorcet and individual rank learning methods , 2009, SIGIR.

[6]  Lei Wang,et al.  Encoding High Dimensional Local Features by Sparse Coding Based Fisher Vectors , 2014, NIPS.

[7]  Xian-Sheng Hua,et al.  Bayesian Visual Reranking , 2011, IEEE Transactions on Multimedia.

[8]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Anil K. Jain,et al.  Image retrieval using color and shape , 1996, Pattern Recognit..

[10]  A.L. Ion,et al.  Semantic Based Image Retrieval Using Relevance Feedback , 2007, EUROCON 2007 - The International Conference on "Computer as a Tool".

[11]  Tao Mei,et al.  Learning to video search rerank via pseudo preference feedback , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[12]  David J. Sheskin,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 1997 .

[13]  Xin-She Yang,et al.  Engineering optimisation by cuckoo search , 2010 .

[14]  Hao Su,et al.  Object Bank: An Object-Level Image Representation for High-Level Visual Recognition , 2014, International Journal of Computer Vision.

[15]  Craig Boutilier,et al.  Learning Mallows Models with Pairwise Preferences , 2011, ICML.

[16]  Reiner Lenz,et al.  Compact colour descriptors for colour-based image retrieval , 2005, Signal Process..

[17]  Heung-Kyu Lee,et al.  Re-ranking algorithm using post-retrieval clustering for content-based image retrieval , 2005, Inf. Process. Manag..

[18]  Ricardo da Silva Torres,et al.  Exploiting clustering approaches for image re-ranking , 2011, J. Vis. Lang. Comput..

[19]  Shaoping Ma,et al.  Relevance feedback in content-based image retrieval: Bayesian framework, feature subspaces, and progressive learning , 2003, IEEE Trans. Image Process..

[20]  Shih-Fu Chang,et al.  Video search reranking through random walk over document-level context graph , 2007, ACM Multimedia.

[21]  Wei Liu,et al.  Noise resistant graph ranking for improved web image search , 2011, CVPR 2011.

[22]  Maria L. Rizzo,et al.  Measuring and testing dependence by correlation of distances , 2007, 0803.4101.

[23]  Hermann Ney,et al.  Learning weighted distances for relevance feedback in image retrieval , 2008, 2008 19th International Conference on Pattern Recognition.

[24]  Bin Zhao,et al.  Sparse Output Coding for Large-Scale Visual Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Javed A. Aslam,et al.  Models for metasearch , 2001, SIGIR '01.

[26]  Ricardo da Silva Torres,et al.  Image Re-ranking and Rank Aggregation Based on Similarity of Ranked Lists , 2011, CAIP.

[27]  Guojun Lu,et al.  Shape-based image retrieval using generic Fourier descriptor , 2002, Signal Process. Image Commun..

[28]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[29]  Peter Kontschieder,et al.  Beyond Pairwise Shape Similarity Analysis , 2009, ACCV.

[30]  Longin Jan Latecki,et al.  Affinity learning on a tensor product graph with applications to shape and image retrieval , 2011, CVPR 2011.

[31]  S. Venkateswarlu,et al.  Fourier Descriptors For Shape Based Image Retrieval , 2013 .

[32]  Tao Mei,et al.  Optimizing Visual Search Reranking via Pairwise Learning , 2011, IEEE Transactions on Multimedia.

[33]  Ming Yang,et al.  Query Specific Rank Fusion for Image Retrieval , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Kriengkrai Porkaew,et al.  Query refinement for multimedia similarity retrieval in MARS , 1999, MULTIMEDIA '99.

[35]  David C. Gibbon,et al.  Relevance Feedback using Support Vector Machines , 2001, ICML.

[36]  Subrahmanyam Murala,et al.  Directional local extrema patterns: a new descriptor for content based image retrieval , 2012, International Journal of Multimedia Information Retrieval.

[37]  Stevan Rudinac,et al.  Exploiting visual reranking to improve pseudo-relevance feedback for spoken-content-based video retrieval , 2009, 2009 10th Workshop on Image Analysis for Multimedia Interactive Services.

[38]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[39]  John Law,et al.  Robust Statistics—The Approach Based on Influence Functions , 1986 .

[40]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[41]  Tao Qin,et al.  A New Probabilistic Model for Rank Aggregation , 2010, NIPS.

[42]  Atilla Baskurt,et al.  Generalizations of angular radial transform for 2D and 3D shape retrieval , 2005, Pattern Recognit. Lett..

[43]  Guojun Lu,et al.  Generic Fourier descriptor for shape-based image retrieval , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[44]  Edward A. Fox,et al.  Combination of Multiple Searches , 1993, TREC.

[45]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Moni Naor,et al.  Rank aggregation methods for the Web , 2001, WWW '01.

[47]  Umberto Straccia,et al.  Web metasearch: rank vs. score based rank aggregation methods , 2003, SAC '03.

[48]  Jing-Yu Yang,et al.  Content-based image retrieval using color difference histogram , 2013, Pattern Recognit..

[49]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[50]  Saeid Belkasim,et al.  Invariant curvature-based Fourier shape descriptors , 2012, J. Vis. Commun. Image Represent..

[51]  Daniel Carlos Guimarães Pedronette,et al.  Unsupervised manifold learning using Reciprocal kNN Graphs in image re-ranking and rank aggregation tasks , 2014, Image Vis. Comput..

[52]  Thomas S. Huang,et al.  Relevance feedback in image retrieval: A comprehensive review , 2003, Multimedia Systems.

[53]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[54]  V. K. Govindan,et al.  Optimizing visual dictionaries for effective image retrieval , 2015, International Journal of Multimedia Information Retrieval.

[55]  Brandeis Marshall,et al.  Applying Aggregation Concepts for Image Search , 2008, 2008 Tenth IEEE International Symposium on Multimedia.

[56]  Xiangyang Wang,et al.  Content-based image retrieval by integrating color and texture features , 2012, Multimedia Tools and Applications.

[57]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[58]  Luc Van Gool,et al.  Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors , 2011, CVPR 2011.

[59]  Samia Nefti-Meziani,et al.  A Comprehensive Review of Swarm Optimization Algorithms , 2015, PloS one.

[60]  Ying Wu,et al.  Object retrieval and localization with spatially-constrained similarity measure and k-NN re-ranking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Marin Ferecatu,et al.  Semantic interactive image retrieval combining visual and conceptual content description , 2007, Multimedia Systems.

[62]  Cordelia Schmid,et al.  Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search , 2008, ECCV.

[63]  Dervis Karaboga,et al.  A comparative study of Artificial Bee Colony algorithm , 2009, Appl. Math. Comput..

[64]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[65]  Subrahmanyam Murala,et al.  Local Tetra Patterns: A New Feature Descriptor for Content-Based Image Retrieval , 2012, IEEE Transactions on Image Processing.

[66]  Henning Müller,et al.  Fusion Techniques for Combining Textual and Visual Information Retrieval , 2010, ImageCLEF.

[67]  Husniza Husni,et al.  A weighted dominant color descriptor for content-based image retrieval , 2013, J. Vis. Commun. Image Represent..

[68]  John R. Smith,et al.  IBM Research TRECVID-2009 Video Retrieval System , 2009, TRECVID.

[69]  Andrew Zisserman,et al.  Near Duplicate Image Detection: min-Hash and tf-idf Weighting , 2008, BMVC.

[70]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).