Colour Image Retrieval and Object Recognition Using the Multimodal Neighbourhood Signature

A novel approach to colour-based object recognition and image retrieval -the multimodal neighbourhood signature- is proposed. Object appearance is represented by colour-based features computed from image neighbourhoods with multi-modal colour density function. Stable invariants are derived from modes of the density function that are robustly located by the mean shift algorithm. The problem of extracting local invariant colour features is addressed directly, without a need for prior segmentation or edge detection. The signature is concise - an image is typically represented by a few hundred bytes, a few thousands for very complex scenes. The algorithm's performance is first tested on a region-based image retrieval task achieving a good (92%) hit rate at a speed of 600 image comparisons per second. The method is shown to operate successfully under changing illumination, viewpoint and object pose, as well as non-rigid object deformation, partial occlusion and the presence of background clutter dominating the scene. The performance of the multimodal neighbourhood signature method is also evaluated on a standard colour object recognition task using a publicly available dataset. Very good recognition performance (average match percentile 99.5%) was achieved in real time (average 0.28 seconds for recognising a single image) which compares favourably with results reported in the literature.