Content-based image retrieval: approaches and trends of the new age

The last decade has witnessed great interest in research on content-based image retrieval. This has paved the way for a large number of new techniques and systems, and a growing interest in associated fields to support such systems. Likewise, digital imagery has expanded its horizon in many directions, resulting in an explosion in the volume of image data required to be organized. In this paper, we discuss some of the key contributions in the current decade related to image retrieval and automated image annotation, spanning 120 references. We also discuss some of the key challenges involved in the adaptation of existing image retrieval techniques to build useful systems that can handle real-world data. We conclude with a study on the trends in volume and impact of publications in the field with respect to venues/journals and sub-topics.

[1]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[3]  Nuno Vasconcelos,et al.  Learning from User Feedback in Image Retrieval Systems , 1999, NIPS.

[4]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[5]  Filippo Menczer,et al.  Feature selection in unsupervised learning via evolutionary search , 2000, KDD '00.

[6]  Qi Tian,et al.  Discriminant-EM algorithm with application to image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[7]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Lei Zhu,et al.  Keyblock: an approach for content-based image retrieval , 2000, ACM Multimedia.

[9]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Robert M. Gray,et al.  Multiresolution image classification by hierarchical modeling with two-dimensional hidden Markov models , 2000, IEEE Trans. Inf. Theory.

[11]  Nuno Vasconcelos,et al.  A probabilistic architecture for content-based image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[12]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[13]  Mihai Datcu,et al.  Interactive learning and probabilistic retrieval in remote sensing image archives , 2000, IEEE Trans. Geosci. Remote. Sens..

[14]  Ingemar J. Cox,et al.  The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments , 2000, IEEE Trans. Image Process..

[15]  Qiang Yang,et al.  A unified framework for semantics and feature based relevance feedback in image retrieval systems , 2000, ACM Multimedia.

[16]  Longin Jan Latecki,et al.  Shape Similarity Measure Based on Correspondence of Visual Parts , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Thomas S. Huang,et al.  Optimizing learning in image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  B. S. Manjunath,et al.  Adaptive nearest neighbor search for relevance feedback in large image databases , 2001, MULTIMEDIA '01.

[19]  Anil K. Jain,et al.  Image classification for content-based indexing , 2001, IEEE Trans. Image Process..

[20]  Thomas S. Huang,et al.  Comparing discriminating transformations and SVM for learning during multimedia retrieval , 2001, MULTIMEDIA '01.

[21]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[22]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[23]  Harry Shum,et al.  Image segmentation by data driven Markov chain Monte Carlo , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[24]  B. S. Manjunath,et al.  An efficient color representation for image retrieval , 2001, IEEE Trans. Image Process..

[25]  john maccoll,et al.  ACM / IEEE Joint Conference on Digital Libraries , 2001 .

[26]  Christian Böhm,et al.  Searching in high-dimensional spaces: Index structures for improving the performance of multimedia databases , 2001, CSUR.

[27]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[28]  Thierry Pun,et al.  Performance evaluation in content-based image retrieval: overview and proposals , 2001, Pattern Recognit. Lett..

[29]  James Ze Wang,et al.  A scalable integrated region-based image retrieval system , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[30]  James Ze Wang,et al.  Unsupervised Multiresolution Segmentation for Images with Low Depth of Field , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Nicu Sebe,et al.  Image retrieval using wavelet-based salient points , 2001, J. Electronic Imaging.

[32]  Erkki Oja,et al.  Self-Organising Maps as a Relevance Feedback Technique in Content-Based Image Retrieval , 2001, Pattern Analysis & Applications.

[33]  Kerry Rodden,et al.  Does organisation by similarity assist image browsing? , 2001, CHI.

[34]  C. A. Murthy,et al.  Unsupervised Feature Selection Using Feature Similarity , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Nozha Boujemaa,et al.  Unsupervised robust clustering for image database categorization , 2002, Object recognition supported by user interaction for service robots.

[36]  Hyeran Byun,et al.  Integrated region-based image retrieval using region's spatial relationships , 2002, Object recognition supported by user interaction for service robots.

[37]  Euripides G. M. Petrakis,et al.  Matching and Retrieval of Distorted and Occluded Shapes Using Dynamic Programming , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Minh N. Do,et al.  Wavelet-based texture retrieval using generalized Gaussian density and Kullback-Leibler distance , 2002, IEEE Trans. Image Process..

[39]  Alberto Del Bimbo,et al.  Three-Dimensional Interfaces for Querying by Example in Content-Based Image Retrieval , 2002, IEEE Trans. Vis. Comput. Graph..

[40]  Zheru Chi,et al.  Fuzzy integral for leaf image retrieval , 2002, 2002 IEEE World Congress on Computational Intelligence. 2002 IEEE International Conference on Fuzzy Systems. FUZZ-IEEE'02. Proceedings (Cat. No.02CH37291).

[41]  John R. Smith,et al.  A study of image retrieval by anchoring , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[42]  Qi Zhang,et al.  Content-Based Image Retrieval Using Multiple-Instance Learning , 2002, ICML.

[43]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[44]  Jake K. Aggarwal,et al.  Retrieval by classification of images containing large manmade objects using perceptual grouping , 2002, Pattern Recognit..

[45]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Erkki Oja,et al.  PicSOM-self-organizing image retrieval with MPEG-7 content descriptors , 2002, IEEE Trans. Neural Networks.

[47]  Rong Jin,et al.  Using a probabilistic source model for comparing images , 2002, Proceedings. International Conference on Image Processing.

[48]  Valérie Gouet-Brunet,et al.  On the robustness of color points of interest for image retrieval , 2002, Proceedings. International Conference on Image Processing.

[49]  Wendi B. Heinzelman,et al.  SPIN-IT: a data centric routing protocol for image retrieval in wireless networks , 2002, Proceedings. International Conference on Image Processing.

[50]  Aleksandra Mojsilovic,et al.  Adaptive image segmentation based on color and texture , 2002, Proceedings. International Conference on Image Processing.

[51]  Yixin Chen,et al.  A Region-Based Fuzzy Feature Matching Approach to Content-Based Image Retrieval , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Thomas S. Huang,et al.  Unifying Keywords and Visual Contents in Image Retrieval , 2002, IEEE Multim..

[53]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Deok-Hwan Kim,et al.  QCluster: relevance feedback using adaptive clustering for content-based image retrieval , 2003, SIGMOD '03.

[55]  R. Manmatha,et al.  A Model for Learning the Semantics of Pictures , 2003, NIPS.

[56]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[57]  Daniel Gatica-Perez,et al.  On image auto-annotation with latent space models , 2003, ACM Multimedia.

[58]  T. Painter,et al.  Retrieval of subpixel snow-covered area and grain size from imaging spectrometer data , 2003 .

[59]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[60]  Koji Nakano,et al.  An image retrieval system using FPGAs , 2003, ASP-DAC '03.

[61]  Thomas S. Huang,et al.  Relevance feedback in image retrieval: A comprehensive review , 2003, Multimedia Systems.

[62]  Edward Y. Chang,et al.  CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines , 2003, IEEE Trans. Circuits Syst. Video Technol..

[63]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[64]  Edward Y. Chang,et al.  Confidence-based dynamic ensemble for image annotation and semantics discovery , 2003, MULTIMEDIA '03.

[65]  Shaoping Ma,et al.  Relevance feedback in content-based image retrieval: Bayesian framework, feature subspaces, and progressive learning , 2003, IEEE Trans. Image Process..

[66]  Ricardo da Silva Torres,et al.  Visual structures for image browsing , 2003, CIKM '03.

[67]  Bir Bhanu,et al.  Active concept learning for image retrieval in dynamic databases , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[68]  Christian Bauckhage,et al.  Combining speech and haptics for intuitive and efficient navigation through image databases , 2003, ICMI '03.

[69]  Jin Li,et al.  On interactive browsing of large images , 2003, IEEE Trans. Multim..

[70]  Kerry Rodden,et al.  How do people manage their digital photographs? , 2003, CHI '03.

[71]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[72]  Thomas S. Huang,et al.  Evaluating group-based relevance feedback for content-based image retrieval , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[73]  I. Andreadis,et al.  Colour histogram content-based image retrieval and hardware implementation , 2003 .

[74]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[75]  Kevin Li,et al.  Faceted metadata for image search and browsing , 2003, CHI '03.

[76]  Kyuseok Shim,et al.  WALRUS: a similarity retrieval algorithm for image databases , 1999, IEEE Transactions on Knowledge and Data Engineering.

[77]  Jianbo Shi,et al.  Segmentation given partial grouping constraints , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[78]  Xiaofei He.  Incremental semi-supervised subspace learning for image retrieval , 2004, MULTIMEDIA '04.

[79]  Michael R. Lyu,et al.  Group-based relevance feedback with support vector machine ensembles , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[80]  Michael R. Lyu,et al.  A novel log-based relevance feedback technique in content-based image retrieval , 2004, MULTIMEDIA '04.

[81]  Wei-Ying Ma,et al.  Learning an image manifold for retrieval , 2004, MULTIMEDIA '04.

[82]  Serge J. Belongie,et al.  Normalized cuts in 3-D for spinal MRI segmentation , 2004, IEEE Transactions on Medical Imaging.

[83]  Bo Zhang,et al.  Relevance feedback in region-based image retrieval , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[84]  Jitendra Malik,et al.  Contour and Texture Analysis for Image Segmentation , 2001, International Journal of Computer Vision.

[85]  James Ze Wang,et al.  The story picturing engine: finding elite images to illustrate a story using mutual reinforcement , 2004, MIR '04.

[86]  R. Sukthankar,et al.  Object-based image retrieval using the statistical structure of images , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[87]  Shree K. Nayar,et al.  Multiresolution histograms and their use for recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[88]  Edward Y. Chang,et al.  Multimodal concept-dependent active learning for image retrieval , 2004, MULTIMEDIA '04.

[89]  Yixin Chen,et al.  Image Categorization by Learning and Reasoning with Regions , 2004, J. Mach. Learn. Res..

[90]  D. Bainbridge,et al.  How people describe their image information needs: a grounded theory analysis of visual arts queries , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..

[91]  Hanqing Lu,et al.  WillHunter: Interactive Image Retrieval with Multilevel Relevance Measurement , 2004, ICPR 2004.

[92]  Nicu Sebe,et al.  Boosting contextual information in content-based image retrieval , 2004, MIR '04.

[93]  Hans Hinterberger,et al.  Content-Based Image Retrieval in Astronomy , 2000, Information Retrieval.

[94]  Jingrui He,et al.  Mean version space: a new active learning method for content-based image retrieval , 2004, MIR '04.

[95]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[96]  Hong-Jiang Zhang,et al.  An efficient and effective region-based image retrieval framework , 2004, IEEE Transactions on Image Processing.

[97]  Robert M. Gray,et al.  Image retrieval using color histograms generated by Gauss mixture vector quantization , 2004, Comput. Vis. Image Underst..

[98]  James Ze Wang,et al.  Stochastic modeling of volume images with a 3-D hidden Markov model , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[99]  Paul A. Viola,et al.  Boosting Image Retrieval , 2004, International Journal of Computer Vision.

[100]  James Ze Wang,et al.  Digital imagery for significant cultural and historical materials , 2005, International Journal on Digital Libraries.

[101]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[102]  Luo Si,et al.  Effective automatic image annotation via a coherent language model and active learning , 2004, MULTIMEDIA '04.

[103]  Thomas S. Huang,et al.  A framework for grid-based image retrieval , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[104]  Wei-Ying Ma,et al.  Locality preserving clustering for image database , 2004, MULTIMEDIA '04.

[105]  Nuno Vasconcelos,et al.  On the efficient evaluation of probabilistic similarity functions for image retrieval , 2004, IEEE Transactions on Information Theory.

[106]  Antoine Geissbühler,et al.  A Review of Content{Based Image Retrieval Systems in Medical Applications { Clinical Bene(cid:12)ts and Future Directions , 2022 .

[107]  Abdolah Chalechale,et al.  Sketch-based image matching Using Angular partitioning , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[108]  Gustavo Carneiro,et al.  Minimum Bayes error features for visual recognition by sequential feature selection and extraction , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[109]  Nuno Vasconcelos,et al.  A multiresolution manifold distance for invariant image similarity , 2005, IEEE Transactions on Multimedia.

[110]  King Ngi Ngan,et al.  A memory learning framework for effective image retrieval , 2005, IEEE Transactions on Image Processing.

[111]  George Economou,et al.  A generic scheme for color image retrieval based on the multivariate Wald-Wolfowitz test , 2005, IEEE Transactions on Knowledge and Data Engineering.

[112]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[113]  Ilaria Bartolini,et al.  WARP: accurate retrieval of shapes using phase of Fourier descriptors and time warping distance , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[114]  Yixin Chen,et al.  CLUE: cluster-based retrieval of images by unsupervised learning , 2005, IEEE Transactions on Image Processing.

[115]  Patrick Baudisch,et al.  Time quilt: scaling up zoomable photo browsers for large, unstructured photo collections , 2005, CHI EA '05.

[116]  Xing Xie,et al.  Learning user interest for image browsing on small-form-factor devices , 2005, CHI.

[117]  Anup Basu,et al.  Image retrieval based on histogram of fractal parameters , 2005, IEEE Transactions on Multimedia.

[118]  Naphtali Rishe,et al.  Content-based image retrieval , 1995, Multimedia Tools and Applications.