Object Re-detection Using SIFT and MPEG-7 Color Descriptors

Information about the occurrence of objects in videos and their interactions conveys an important part of the semantics of audio-visual content and can be used to narrow the semantic gap in video analysis, retrieval and summarization. Object re-detection, which aims at finding occurrences of specific objects in a single video or a collection of still images and videos, is an object identification problem and can thus be more satisfactorily solved than a general object recognition problem. As structural information and color information are often complementary, we propose a combined object re-detection approach using SIFT and MPEG-7 color descriptors extracted around the same interest points. We evaluate the approach on two different data sets and show that the MPEG-7 ColorLayout descriptor performs best of the tested color descriptors and that the joint approach yields better results than the use of SIFT or color descriptors only.