Searching images with MPEG-7 (& MPEG-7-like) Powered Localized dEscriptors: The SIMPLE answer to effective Content Based Image Retrieval

In this paper we propose and evaluate a new technique that localizes the description ability of the well established MPEG-7 and MPEG-7-like global descriptors. We employ the SURF detector to define salient image patches of blob-like textures and use the MPEG-7 Scalable Color (SC), Color Layout (CL) and Edge Histogram (EH) descriptors and the global MPEG-7-like Color and Edge Directivity Descriptor (CEDD), to produce the final local features' vectors. In order to test the new descriptors in the most straightforward fashion, we use the Bag-Of-Visual-Words framework for indexing and retrieval. The experimental results conducted on two different benchmark databases with varying codebook sizes, revealed an astonishing boost in the retrieval performance of the proposed descriptors compared both to their own performance (in their original form) and to other state-of-the-art methods of local and global descriptors. Open-source implementation of the proposed descriptors is available in c#, Java and MATLAB.

[1]  Yiannis S. Boutalis,et al.  CEDD: Color and Edge Directivity Descriptor: A Compact Descriptor for Image Indexing and Retrieval , 2008, ICVS.

[2]  Yiannis S. Boutalis,et al.  Co.Vi.Wo.: Color Visual Words Based on Non-Predefined Size Codebooks , 2013, IEEE Transactions on Cybernetics.

[3]  Christopher G. Harris,et al.  3D positional integration from image sequences , 1988, Image Vis. Comput..

[4]  Yiannis S. Boutalis,et al.  FCTH: Fuzzy Color and Texture Histogram - A Low Level Feature for Accurate Image Retrieval , 2008, 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services.

[5]  Gerald Schaefer,et al.  UCID: an uncompressed color image database , 2003, IS&T/SPIE Electronic Imaging.

[6]  Remco C. Veltkamp,et al.  Fixed partitioning and salient points with MPEG-7 cluster correlograms for image categorization , 2010, Pattern Recognit..

[7]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[8]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[9]  Sven Loncaric,et al.  A survey of shape analysis techniques , 1998, Pattern Recognit..

[10]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[12]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[13]  Stefan M. Rüger,et al.  Evaluation of Texture Features for Content-Based Image Retrieval , 2004, CIVR.

[14]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[15]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[16]  Pietro Perona,et al.  Automatic discovery of image families: Global vs. local features , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[17]  George R. Thoma,et al.  A medical image retrieval framework in correlation enhanced visual concept feature space , 2009, 2009 22nd IEEE International Symposium on Computer-Based Medical Systems.

[18]  J.-P. Renno,et al.  Evaluation of MPEG7 color descriptors for visual surveillance retrieval , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[19]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[20]  Yannis Avrithis,et al.  Fuzzy support vector machines for image classification fusing MPEG-7 visual descriptors , 2005 .

[21]  Mathias Lux,et al.  Lire: lucene image retrieval: an extensible java CBIR library , 2008, ACM Multimedia.

[22]  George R. Thoma,et al.  A classification-driven similarity matching framework for retrieval of biomedical images , 2010, MIR '10.

[23]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[24]  Cyrus Shahabi,et al.  Image retrieval by shape: a comparative study , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[25]  Feng Xu,et al.  Evaluation and comparison of texture descriptors proposed in MPEG-7 , 2006, J. Vis. Commun. Image Represent..

[26]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[27]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[28]  Yannis Avrithis,et al.  Fusing MPEG-7 Visual Descriptors for Image Classification , 2005, ICANN.

[29]  Yiannis S. Boutalis,et al.  Content based radiology image retrieval using a fuzzy rule based scalable composite descriptor , 2009, Multimedia Tools and Applications.

[30]  Yiannis S. Boutalis,et al.  Mean Normalized Retrieval Order (MNRO): a new content-based image retrieval performance measure , 2014, Multimedia Tools and Applications.

[31]  Mathias Lux,et al.  Img(Rummager): An Interactive Content Based Image Retrieval System , 2009, 2009 Second International Workshop on Similarity Search and Applications.

[32]  Kristin J. Dana,et al.  Compact representation of bidirectional texture functions , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[33]  Hermann Ney,et al.  Features for image retrieval: an experimental comparison , 2008, Information Retrieval.

[34]  Roland Siegwart,et al.  Introduction to Autonomous Mobile Robots , 2004 .

[35]  Yiannis S. Boutalis,et al.  Golden retriever: a Java based open source image retrieval engine , 2013, MM '13.

[36]  Patrick J. Flynn,et al.  A 20th Anniversary Survey: Introduction to 'Content-Based Image Retrieval at the End of the Early Years' , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Guojun Lu,et al.  Review of shape representation and description techniques , 2004, Pattern Recognit..

[38]  Claudio Gennaro,et al.  Combining local and global visual feature similarity using a text search engine , 2011, 2011 9th International Workshop on Content-Based Multimedia Indexing (CBMI).

[39]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[40]  Xi Li,et al.  Ranking consistency for image matching and object retrieval , 2014, Pattern Recognit..

[41]  Ricardo da Silva Torres,et al.  Color Descriptors for Web Image Retrieval: A Comparative Study , 2008, 2008 XXI Brazilian Symposium on Computer Graphics and Image Processing.

[42]  Alan F. Smeaton,et al.  Combining image descriptors to effectively retrieve events from visual lifelogs , 2008, MIR '08.

[43]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .