Spatial Color Indexing and Applications

We define a new image feature called the color correlogram and use it for image indexing and comparison. This feature distills the spatial correlation of colors and when computed efficiently, turns out to be both effective and inexpensive for content-based image retrieval. The correlogram is robust in tolerating large changes in appearance and shape caused by changes in viewing position, camera zoom, etc. Experimental evidence shows that this new feature outperforms not only the traditional color histogram method but also the recently proposed histogram refinement methods for image indexing/retrieval. We also provide a technique to cut down the storage requirement of the correlogram so that it is the same as that of histograms, with only negligible performance penalty compared to the original correlogram.We also suggest the use of color correlogram as a generic indexing tool to tackle various problems arising from image retrieval and video browsing. We adapt the correlogram to handle the problems of image subregion querying, object localization, object tracking, and cut detection. Experimental results again suggest that the color correlogram is more effective than the histogram for these applications, with insignificant additional storage or processing cost.

[1]  D. Marr,et al.  Representation and recognition of the spatial organization of three-dimensional shapes , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[2]  C.-C. Jay Kuo,et al.  Color distribution analysis and quantization for image retrieval , 1996, Electronic Imaging.

[3]  Ramin Zabih,et al.  Histogram Re nement for Content-Based Image RetrievalGreg , 1996 .

[4]  G. J. G. Upton,et al.  Spatial data Analysis by Example , 1985 .

[5]  David A. Forsyth,et al.  Finding Naked People , 1996, ECCV.

[6]  Yan Gong,et al.  Intelligent image databases - towards advanced image retrieval , 1997, The Kluwer international series in engineering and computer science.

[7]  Clark F. Olson,et al.  Object Recognition Using Subspace Methods , 1996, ECCV.

[8]  Tat-Seng Chua,et al.  An integrated color-spatial approach to content-based image retrieval , 1995, MULTIMEDIA '95.

[9]  Brian V. Funt,et al.  Color Constant Color Indexing , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  John S. Boreczky,et al.  Comparison of video shot boundary detection techniques , 1996, J. Electronic Imaging.

[11]  Ramin Zabih,et al.  Histogram refinement for content-based image retrieval , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[12]  Jiri Matas,et al.  On representation and matching of multi-coloured objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[13]  Vijay V. Raghavan,et al.  Content-Based Image Retrieval Systems - Guest Editors' Introduction , 1995, Computer.

[14]  Yihong Gong,et al.  An image database system with content capturing and fast image indexing abilities , 1994, 1994 Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[15]  Hayit Greenspan,et al.  Finding Pictures of Objects in Large Collections of Images , 1996, Object Representation in Computer Vision.

[16]  Ramin Zabih,et al.  Comparing images using color coherence vectors , 1997, MULTIMEDIA '96.

[17]  Glenn Healey,et al.  Combining color and geometric information for the illumination invariant recognition of 3-D objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[18]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[19]  Lawrence G. Roberts,et al.  Machine Perception of Three-Dimensional Solids , 1963, Outstanding Dissertations in the Computer Sciences.

[20]  Daphna Weinshall,et al.  Condensing image databases when retrieval is based on non-metric distances , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[21]  R.M. Haralick,et al.  Statistical and structural approaches to texture , 1979, Proceedings of the IEEE.

[22]  Alex Pentland,et al.  Photobook: Content-based manipulation of image databases , 1996, International Journal of Computer Vision.

[23]  Gérard G. Medioni,et al.  Finding Waldo, or Focus of Attention Using Local Color Information , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Jing Huang,et al.  Spatial Color Indexing and Applications , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[25]  MedioniGérard,et al.  Finding Waldo, or Focus of Attention Using Local Color Information , 1995 .

[26]  James Lee Hafner,et al.  Efficient Color Histogram Indexing for Quadratic Form Distance Functions , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[28]  Hiroshi Murase,et al.  Focused color intersection with efficient searching for object detection and image retrieval , 1996, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems.

[29]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Shih-Fu Chang,et al.  Tools and techniques for color image retrieval , 1996, Electronic Imaging.

[31]  Tanveer F. Syeda-Mahmood.  Data and Model-Driven Selection using Color Regions , 1992, ECCV.

[32]  Azriel Rosenfeld,et al.  Using probabilistic domain knowledge to reduce the expected computational cost of template matching , 1990, Comput. Vis. Graph. Image Process..

[33]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[34]  Peter Stanchev,et al.  Content-Based Image Retrieval Systems , 2001 .

[35]  Markus A. Stricker,et al.  Color indexing with weak spatial constraints , 1996, Electronic Imaging.

[36]  David Haussler,et al.  Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications , 1992, Inf. Comput..

[37]  T. John Stonham,et al.  Content-based image retrieval using color tuple histograms , 1996, Electronic Imaging.

[38]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.

[39]  Tanveer F. Syeda-Mahmood,et al.  Indexing colored surfaces in images , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[40]  Michael Stonebraker,et al.  Chabot: Retrieval from a Relational Database of Images , 1995, Computer.

[41]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[42]  Jing Huang,et al.  Combining supervised learning with color correlograms for content-based image retrieval , 1997, MULTIMEDIA '97.

[43]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[44]  Rajesh P. N. Rao,et al.  Object indexing using an iconic sparse distributed memory , 1995, Proceedings of IEEE International Conference on Computer Vision.

[45]  T. Coburn.  Spatial Data Analysis by Example , 1991 .

[46]  Ramin Zabih,et al.  Comparing images using joint histograms , 1999, Multimedia Systems.

[47]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[48]  Michael J. Swain,et al.  The capacity of color histogram indexing , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[49]  W. Eric L. Grimson,et al.  Localizing Overlapping Parts by Searching the Interpretation Tree , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  John S. Boreczky,et al.  Comparison of video shot boundary detection techniques , 1996, Electronic Imaging.

[51]  Gérard G. Medioni,et al.  Finding Waldo, or focus of attention using local color information , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Tim Ellis,et al.  Using Colour Templates for Target Identification and Tracking , 1992 .

[53]  Tanveer F. Syeda-Mahmood,et al.  Data and Model-Driven Selection Using Color Regions , 1992, International Journal of Computer Vision.

[54]  Ingemar J. Cox,et al.  PicHunter: Bayesian relevance feedback for image retrieval , 1996, Proceedings of 13th International Conference on Pattern Recognition.