Colour-based Image Retrieval from Video Sequences

The multimodal neighbourhood signature (MNS) algorithm represents local object appearance by stable colour-based invariants efficiently computed from image neighbourhoods with multimodal colour density function. Local feature extraction facilitates region-based interactive query specification and computation of illumination invariant features. The method allows for fast signature matching and supports retrieval of objects covering only a fraction of the database image. MNS signatures are generally compact and storage requirements are typically a few hundred bytes per image. In this paper, the proposed algorithm is tested on a challenging region-based image retrieval task, searching for objects in a sports video sequence. Image regions depicting objects of typical interest in a sports image database are delineated by the users and used as queries. Acceptable retrieval results are presented for a number of experiments. Relevant images were retrieved regardless of background clutter dominating the scene, partial occlusion and/or non-rigid object deformation. Finally, the suitability of the approach is investigated in the context of webbased applications where efficient signature creation, good matching speed and minimal storage requirements are required.

[1]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[2]  G. Healey,et al.  Global color constancy: recognition of objects by use of illumination-invariant properties of color distributions , 1994 .

[3]  Graham D. Finlayson,et al.  Colour object recognition , 1992 .

[4]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[5]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Brian V. Funt,et al.  Color Constant Color Indexing , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Arnold W. M. Smeulders,et al.  Color-based object recognition , 1997, Pattern Recognit..

[8]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[9]  J. Kittler,et al.  SELECTING FEATURES FOR NEURAL NETWORKS TO AID AN ICONIC SEARCHTHROUGH AN IMAGE , 1997 .

[10]  Bruce A. Draper,et al.  FOCUS: Searching for multi-colored objects in a diverse image database , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Dorin Comaniciu,et al.  Mean shift analysis and applications , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Sang Uk Lee,et al.  Color image retrieval using hybrid graph representation , 1999, Image Vis. Comput..

[13]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  W. Eric L. Grimson,et al.  Using Photometric Invariants for 3D Object Recognition , 1998, Comput. Vis. Image Underst..

[15]  Luc Van Gool,et al.  Recognizing color patterns irrespective of viewpoint and illumination , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[16]  Shih-Fu Chang,et al.  Integrated spatial and feature image query , 1999, Multimedia Systems.

[17]  J. Kittler,et al.  Performance Evaluation of the Multi-modal Neighbourhood Signature Method for Colour Object Recognition , 2000 .

[18]  Ronen Basri,et al.  Comparing images under variable illumination , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[19]  Brian V. Funt,et al.  Is Machine Colour Constancy Good Enough? , 1998, ECCV.