Illumination-invariant image retrieval and video segmentation

Abstract Images or videos may be imaged under different illuminants than models in an image or video proxy database. Changing illumination color in particular may confound recognition algorithms based on color histograms or video segmentation routines based on these. Here we show that a very simple method of discounting illumination changes is adequate for both image retrieval and video segmentation tasks. We develop a feature vector of only 36 values that can also be used for both these objectives as well as for retrieval of video proxy images from a database. The new image metric is based on a color-channel-normalization step, followed by reduction of dimensionality by going to a chromaticity space. Treating chromaticity histograms as images , we perform an effective low-pass filtering of the histogram by first reducing its resolution via a wavelet-based compression and then by a DCT transformation followed by zonal coding. We show that the color constancy step – color band normalization – can be carried out in the compressed domain for images that are stored in compressed form, and that only a small amount of image information need be decompressed in order to calculate the new metric. The new method performs better than previous methods tested for image or texture recognition and operates entirely in the compressed domain, on feature vectors. Apart from achieving illumination invariance for video segmentation, so that, e.g.an actor stepping out of a shadow does not trigger the declaration of a false cut, the metric reduces all videos to a uniform scale. Thus thresholds can be developed for a training set of videos and applied to any new video, including streaming video, for segmentation as a one-pass operation.

[1]  Brian V. Funt,et al.  Color Constant Color Indexing , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Takafumi Miyatake,et al.  IMPACT: an interactive natural-motion-picture dedicated multimedia authoring system , 1991, CHI.

[3]  M. S. Drew,et al.  Color constancy - Generalized diagonal transforms suffice , 1994 .

[4]  Michel Barlaud,et al.  Image coding using wavelet transform , 1992, IEEE Trans. Image Process..

[5]  J. Cohen,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulas , 1968 .

[6]  Jie Wei,et al.  On illumination invariance in color object recognition , 1998, Pattern Recognit..

[7]  Glenn Healey,et al.  Combining color and geometric information for the illumination invariant recognition of 3-D objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[8]  Daniel Berwick,et al.  A chromaticity space for specularity, illumination color- and illumination pose-invariant 3-D object recognition , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[9]  Nilesh V. Patel,et al.  Statistical approach to scene change detection , 1995, Electronic Imaging.

[10]  Ramesh C. Jain,et al.  Illumination independent change detection for real world image sequences , 1989, Comput. Vis. Graph. Image Process..

[11]  Brian V. Funt,et al.  Color Angular Indexing , 1996, ECCV.

[12]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[13]  G. Healey,et al.  Illumination-invariant recognition of texture in color images , 1995 .

[14]  Minerva M. Yeung,et al.  Efficient matching and clustering of video shots , 1995, Proceedings., International Conference on Image Processing.

[15]  Rameshjain.  Illumination independent Change Detection for Real World Image Sequences , 1989 .

[16]  B. S. Manjunath,et al.  NeTra: A toolbox for navigating large image databases , 1997, Proceedings of International Conference on Image Processing.

[17]  Jie Wei,et al.  Illumination-invariant color object recognition via compressed chromaticity histograms of color-channel-normalized images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[18]  Glenn Healey,et al.  Using color for geometry-insensitive segmentation , 1989 .

[19]  G D Finlayson,et al.  Spectral sharpening: sensor transformations for improved color constancy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[20]  Jie Wei,et al.  Illumination-invariant video segmentation by hierarchical robust thresholding , 1997, Electronic Imaging.

[21]  Carlos F. Borges.  Trichromatic approximation method for surface illumination , 1991 .

[22]  John P. Oakley,et al.  Storage and Retrieval for Image and Video Databases , 1993 .

[23]  Glenn Healey,et al.  Illumination and geometry invariant recognition of texture in color images , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  John S. Boreczky,et al.  Comparison of video shot boundary detection techniques , 1996, Electronic Imaging.

[25]  Arding Hsu,et al.  Feature management for large video databases , 1993, Electronic Imaging.

[26]  Boon-Lock Yeo,et al.  A unified approach to temporal segmentation of motion JPEG and MPEG compressed video , 1995, Proceedings of the International Conference on Multimedia Computing and Systems.

[27]  Glenn Healey,et al.  The illumination-invariant recognition of color texture , 1995, Proceedings of IEEE International Conference on Computer Vision.

[28]  Akio Nagasaka,et al.  Automatic Video Indexing and Full-Video Search for Object Appearances , 1991, VDB.

[29]  G. Healey,et al.  Retrieving Multispectral Satellite Images Using Physics-Based Invariant Representations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..