From Text to Images: Weighting Schemes for Image Retrieval

Bags of visual words are the most studied image description technique in the last years. This representation of images raised new possibilities as well as new research issues. In particular, it is important to automatically determine which visual words are the most relevant to describe the images, and which ones should be ignored. This issue is a classical problem of textual information retrieval, usually addressed by the use of weighting schemes. In this paper, the most common weighting schemes from text retrieval are applied to the case of visual word-based retrieval. New weighting schemes are also proposed, and several Minkowski-like distances are tested. The experiments are performed on four different datasets that correspond to two different retrieval tasks; it allows us to bring to light some properties of visual words and weighting schemes. This study results in several findings. It first shows that the optimal setting for distances and weighting schemes depends on the nature of the visual content of the images considered. Especially, raw frequency can be the most effective weight when dealing with complex datasets; it questions the habit to systematically use the tf . idf weighting scheme. It also shows that weighting schemes and Minkowski distances have similar effect and should be used together in a consistent way. Based on these findings, general guidelines for the choice of distances and weighting schemes are proposed.

[1]  Cordelia Schmid,et al.  Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search , 2008, ECCV.

[2]  Cordelia Schmid,et al.  A contextual dissimilarity measure for accurate and efficient image search , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Hans Peter Luhn,et al.  The Automatic Creation of Literature Abstracts , 1958, IBM J. Res. Dev..

[4]  Michael D. Gordon,et al.  A utility theoretic examination of the probability ranking principle in information retrieval , 1991, J. Am. Soc. Inf. Sci..

[5]  Stephen E. Robertson,et al.  Okapi at TREC-3 , 1994, TREC.

[6]  Hong Chen,et al.  Implementation of software router embedded PID AQM algorithm in the communication test network , 2010, 2010 International Conference on Computer, Mechatronics, Control and Electronic Engineering.

[7]  Stephen P. Harter,et al.  A probabilistic approach to automatic keyword indexing , 1974 .

[8]  Andrew Zisserman,et al.  Scene Classification Via pLSA , 2006, ECCV.

[9]  Stephen E. Robertson,et al.  A probabilistic model of information retrieval: development and comparative experiments - Part 2 , 2000, Inf. Process. Manag..

[10]  Chong-Wah Ngo,et al.  Towards optimal bag-of-features for object categorization and semantic video retrieval , 2007, CIVR '07.

[11]  Wei Wang,et al.  An Overview to FPGA Device Design Technologies , 2010 .

[12]  Chong-Wah Ngo,et al.  Evaluating bag-of-visual-words representations in scene classification , 2007, MIR '07.

[13]  Li Xin-guo.  Algorithm Research of AQM Based on Congestion Control , 2007 .

[14]  Alexander L. Fradkov,et al.  Adaptive tuning of feedback gain in time-delayed feedback control. , 2011, Chaos.

[15]  George E. Andrews,et al.  The lost notebook and other unpublished papers , 1988 .

[16]  T. Kavitha,et al.  Security Vulnerabilities In Wireless Sensor Networks: A Survey , 2010 .

[17]  Toshimichi Saito,et al.  Grouping synchronization in a pulse-coupled network of chaotic spiking oscillators , 2004, IEEE Transactions on Neural Networks.

[18]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[19]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[20]  M. Kikuchi,et al.  Robustness Leads Close to the Edge of Chaos in Coupled Map Networks: toward the understanding of biological networks , 2012, 1205.3944.

[21]  Cordelia Schmid,et al.  Local Grayvalue Invariants for Image Retrieval , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Pierre Tirilly,et al.  A review of weighting schemes for bag of visual words image retrieval , 2009 .

[23]  Kang Yan-lin.  PID Control System Design Based on AVR Microprocessor , 2010 .

[24]  Stefanie Nowak,et al.  New Strategies for Image Annotation: Overview of the Photo Annotation Task at ImageCLEF 2010 , 2010, CLEF.

[25]  Xiaolin Chang,et al.  The effects of AQM on the performance of Assured Forwarding Service , 2006, Comput. Commun..

[26]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[27]  S. Robertson.  The probability ranking principle in IR , 1997 .

[28]  Chong-Wah Ngo,et al.  Visual word proximity and linguistics for semantic video indexing and near-duplicate retrieval , 2009, Comput. Vis. Image Underst..

[29]  James C. French,et al.  Using the triangle inequality to reduce the number of comparisons required for similarity-based retrieval , 1996, Electronic Imaging.

[30]  Frédéric Jurie,et al.  Latent mixture vocabularies for object categorization and segmentation , 2009, Image Vis. Comput..

[31]  C. J. van Rijsbergen,et al.  Probabilistic models of information retrieval based on measuring the divergence from randomness , 2002, TOIS.

[32]  Cordelia Schmid,et al.  TagProp: Discriminative metric learning in nearest neighbor models for image auto-annotation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[33]  C. Schmid,et al.  On the burstiness of visual elements , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[36]  Shu Feng,et al.  Chaos-Based Physical Layer Design for WSN Applications , 2013 .

[37]  Shuai Chen,et al.  Chaos block cipher for wireless sensor network , 2008, Science in China Series F: Information Sciences.

[38]  Frédéric Jurie,et al.  Latent mixture vocabularies for object categorization and segmentation , 2006, Image Vis. Comput..

[39]  Toshimichi Saito,et al.  Basic dynamics from a pulse-coupled network of autonomous integrate-and-fire chaotic circuits , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[40]  Wen Gao,et al.  Effective and efficient object-based image retrieval using visual phrases , 2006, MM '06.

[41]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[42]  Stefan M. Rüger,et al.  Fractional Distance Measures for Content-Based Image Retrieval , 2005, ECIR.

[43]  Arantxa Etxeverria.  The Origins of Order , 1993 .

[44]  Ying Liu,et al.  A chaos-based secure cluster protocol for wireless sensor networks , 2008, Kybernetika.

[45]  James Allan,et al.  Automatic Retrieval With Locality Information Using SMART , 1992, TREC.

[46]  Nen Saito,et al.  Probability of graphs with large spectral gap by multicanonical Monte Carlo , 2011, Comput. Phys. Commun..

[47]  Hidehiro Nakano,et al.  Data Gathering Scheme Using Chaotic Pulse-Coupled Neural Networks for Wireless Sensor Networks , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[48]  Mohammad Haeri,et al.  Implementation of MPC as an AQM controller , 2010, Comput. Commun..

[49]  X. Liao,et al.  A novel block cryptosystem based on iterating a chaotic map , 2006 .

[50]  Pierre Tirilly,et al.  Distances and weighting schemes for bag of visual words image retrieval , 2010, MIR '10.

[51]  Judith Kelner,et al.  Evaluation of security mechanisms in wireless sensor networks , 2005, 2005 Systems Communications (ICW'05, ICHSN'05, ICMCS'05, SENET'05).

[52]  Karen Spärck Jones.  A statistical interpretation of term specificity and its application in retrieval , 2021, J. Documentation.

[53]  Toshimichi Saito,et al.  Basic dynamics from a pulse-coupled network of autonomous integrate-and-fire chaotic circuits , 2002, IEEE Trans. Neural Networks.

[54]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Stephen E. Robertson,et al.  A probabilistic model of information retrieval: development and comparative experiments - Part 1 , 2000, Inf. Process. Manag..

[56]  Xiaojing Wang,et al.  Simulation and Implementation of Adaptive Fuzzy PID , 2014, J. Networks.

[57]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Jean Martinet.  From text vocabularies to visual vocabularies what basis? , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[59]  Xin Chen,et al.  Spatial Weighting for Bag-of-Visual-Words and Its Application in Content-Based Image Retrieval , 2009, PAKDD.

[60]  Yuan Shao-qian.  Chaotic Analysis of Internet Traveling Time , 2008 .

[61]  Pierre Tirilly,et al.  Language modeling for bag-of-visual words image categorization , 2008, CIVR '08.

[62]  Cor J. Veenman,et al.  Visual Word Ambiguity , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  D. R. Frey,et al.  Chaotic digital encoding: an approach to secure communication , 1993 .

[64]  L. Hongyan,et al.  Internet Time-delay Prediction Based on Autoregressive and Neural Network Model , 2006, 2006 International Conference on Communications, Circuits and Systems.

[65]  L. Kocarev,et al.  Logistic map as a block encryption algorithm , 2001 .

[66]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[67]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[68]  Michael Isard,et al.  Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[69]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[70]  Charu C. Aggarwal,et al.  On the Surprising Behavior of Distance Metrics in High Dimensional Spaces , 2001, ICDT.

[71]  Stephen P. Harter,et al.  A probabilistic approach to automatic keyword indexing. Part I. On the Distribution of Specialty Words in a Technical Literature , 1975, J. Am. Soc. Inf. Sci..

[72]  Ellen M. Voorhees,et al.  Evaluating Evaluation Measure Stability , 2000, SIGIR 2000.

[73]  Stephen E. Robertson,et al.  Okapi at TREC-4 , 1995, TREC.

[74]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[75]  Wang Ya-jun.  FPGA hardware implementation scheme for AQM algorithm , 2013 .

[76]  Florent Perronnin,et al.  Fisher Kernels on Visual Vocabularies for Image Categorization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[77]  Fabrizio Sebastiani,et al.  An Analysis of the Relative Hardness of Reuters-21578 Subsets , 2003 .

[78]  Tao Tao,et al.  A formal study of information retrieval heuristics , 2004, SIGIR '04.

[79]  Sun Jinsheng,et al.  Intelligent PID congestion control algorithm , 2004 .