Adaptive image retrieval based on the spatial organization of colors

This work proposes to compare the spatial organization of colors between images through a global optimization procedure relying on the Earth Mover's Distance. The resulting distance is applied to image retrieval. Unlike most region-based retrieval systems, no segmentation of images is needed for the query. We then address the decision stage of the retrieval, that is the problem of automatically deciding which images from a database match a query. To this aim, we make use of an a contrario method. Two images are matched if their proximity is unlikely to be due to chance; more precisely, a matching threshold on distances is computed by controlling the average number of false matchings in an unsupervised way. This threshold is adaptive, yielding different numbers of result images depending on the query and the database.

[1]  Hsin-Liang Chen,et al.  An analysis of image retrieval tasks in the field of art history , 2001, Inf. Process. Manag..

[2]  Hayit Greenspan,et al.  Context-based image modelling , 2002, Object recognition supported by user interaction for service robots.

[3]  Yann Gousseau.  Comparaison de la composition de deux images, et application à la recherche automatique , 2003 .

[4]  Michael Lindenbaum,et al.  An Integrated Model for Evaluating the Amount of Data Required for Reliable Recognition , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Rainer E. Burkard,et al.  Linear Assignment Problems and Extensions , 1999, Handbook of Combinatorial Optimization.

[6]  Lionel Moisan,et al.  Meaningful Alignments , 2000, International Journal of Computer Vision.

[7]  Ira Assent,et al.  Approximation Techniques for Indexing the Earth Mover’s Distance in Multimedia Databases , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[8]  W. Eric L. Grimson,et al.  On the Verification of Hypothesized Matches in Model-Based Recognition , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Hans Burkhardt,et al.  Improvement of histogram-based image retrieval and classification , 2002, Object recognition supported by user interaction for service robots.

[10]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[11]  Hubert Konik,et al.  Color Coarse Segmentation and Regions Selection For Similar Images Retrieval , 2002, CGIV.

[12]  Shi-Kuo Chang,et al.  Iconic Indexing by 2-D Strings , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Patrick Lambert,et al.  Image Retrieval using Spatial Chromatic Histograms , 2004, CGIV.

[14]  Mauro Dell'Amico,et al.  Assignment Problems , 1998, IFIP Congress: Fundamentals - Foundations of Computer Science.

[15]  Yuchun Fang,et al.  An interactive system for mental face retrieval , 2005, MIR '05.

[16]  Cordelia Schmid,et al.  Coloring Local Feature Extraction , 2006, ECCV.

[17]  Yann Gousseau,et al.  Spatial color image retrieval without segmentation using thumbnails and the Earth Mover's Distance , 2006, CGIV.

[18]  Alberto Del Bimbo,et al.  Weighted walkthroughs between extended entities for retrieval by spatial arrangement , 2003, IEEE Trans. Multim..

[19]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[20]  Howard Greisdorf.  Information Seeking Behaviour in Image Retrieval: VISOR I Final Report , 2002, J. Documentation.

[21]  Michael S. Lew,et al.  Principles of Visual Information Retrieval , 2001, Advances in Pattern Recognition.

[22]  Luis Álvarez,et al.  The Size of Objects in Natural and Artificial Images , 1999 .

[23]  Mauro Dell'Amico,et al.  8. Quadratic Assignment Problems: Algorithms , 2009 .

[24]  Bo Zhang,et al.  An effective region-based image retrieval framework , 2002, MULTIMEDIA '02.

[25]  Kanad K. Biswas,et al.  Region-based image retrieval using integrated color, shape, and location index , 2004, Comput. Vis. Image Underst..

[26]  Pamela Briggs,et al.  Image Retrieval Interfaces: A User Perspective , 2004, CIVR.

[27]  Yann Gousseau,et al.  The dead leaves model: a general tessellation modeling occlusion , 2006, Advances in Applied Probability.

[28]  Markus A. Stricker,et al.  Color indexing with weak spatial constraints , 1996, Electronic Imaging.

[29]  Bo Zhang,et al.  An efficient and effective region-based image retrieval framework , 2004, IEEE Transactions on Image Processing.

[30]  G. Wyszecki,et al.  Color Science Concepts and Methods , 1982 .

[31]  Yann Gousseau,et al.  An A Contrario Decision Method for Shape Element Recognition , 2006, International Journal of Computer Vision.

[32]  Lionel Moisan,et al.  Edge Detection by Helmholtz Principle , 2001, Journal of Mathematical Imaging and Vision.

[33]  Haibin Ling,et al.  An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Yann Gousseau,et al.  Modeling Occlusion and Scaling in Natural Images , 2007, Multiscale Model. Simul..

[35]  W. Eric L. Grimson,et al.  Configuration based scene classification and image indexing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  J. Davenport.  Editor , 1960 .

[37]  Gunther Heidemann,et al.  Combining spatial and colour information for content based image retrieval , 2004, Comput. Vis. Image Underst..

[38]  Ramin Zabih,et al.  Comparing images using joint histograms , 1999, Multimedia Systems.

[39]  Ying Liu,et al.  Region-Based Image Retrieval with High-Level Semantic Color Names , 2005, 11th International Multimedia Modelling Conference.

[40]  Jean-Michel Morel,et al.  From Gestalt Theory to Image Analysis: A Probabilistic Approach , 2007 .

[41]  D. R. Fulkerson,et al.  Solving a Transportation Problem , 1956 .

[42]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[43]  Tat-Seng Chua,et al.  An integrated color-spatial approach to content-based image retrieval , 1995, MULTIMEDIA '95.

[44]  W D Wright,et al.  Color Science, Concepts and Methods. Quantitative Data and Formulas , 1967 .

[45]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Matthew Anderson,et al.  Proposal for a Standard Default Color Space for the Internet - sRGB , 1996, CIC.

[47]  John R. Smith,et al.  Image Classification and Querying Using Composite Region Templates , 1999, Comput. Vis. Image Underst..

[48]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[49]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Jing Huang,et al.  Spatial Color Indexing and Applications , 2004, International Journal of Computer Vision.

[51]  Pamela Briggs,et al.  A Taxonomy of the Image: On the Classification of Content for Image Retrieval , 2003 .

[52]  Gregory Grefenstette,et al.  Evaluating Content Based Image Retrieval Techniques with the One Million Images CLIC TestBed , 2005, WEC.

[53]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Shih-Fu Chang,et al.  Tools and techniques for color image retrieval , 1996, Electronic Imaging.

[55]  Alberto Del Bimbo,et al.  Visual information retrieval , 1999 .

[56]  Raimondo Schettini,et al.  Image Indexing and Retrieval Using Spatial Chromatic Histograms and Signatures , 2002, CGIV.

[57]  Farida Cheriet.  A novel collaborative website and artworks database management system for artist-run centers , 2007 .

[58]  Peter J. Bickel,et al.  The Earth Mover's distance is the Mallows distance: some insights from statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[59]  David Mumford,et al.  Occlusion Models for Natural Images: A Statistical Study of a Scale-Invariant Dead Leaves Model , 2004, International Journal of Computer Vision.