Large vocabulary quantization for searching instances from videos

A very promising application involving video collections is to search for relevant video segments from a video database when given few visual examples of the specific instance, e.g. a person, object, or place. However, this problem is difficult due to the lighting variations, different viewpoints, partial occlusion, and large changes in appearance. In this paper, we focus on a kind of restricted instance searching task, where the region of a specific instance to be searched for is manually labeled on each query image. We formulate this problem in a large vocabulary quantization based Bag-of-Words framework, while putting more research emphasis on investigating to what extent we can benefit from these labeled instance regions. The contribution of this paper mainly lies in two aspects: first, we proposed an algorithm for instance search that outperformed all submissions on the instance search dataset TRECVID 2011. Secondly, after thoroughly analyzing the experiment results, we show that our top performance is mainly due to similar scene retrieval, instead of the same instance search. This observation reveals that in the current dataset background is more dominated than instance, and it also suggests that a promising direction in which to further improve the current algorithm, which may also be the breakthrough for achieving this challenge, is to investigate more about how to truly take advantage of additional labeled instance regions. We believe our research opens a window for future new methods for searching instance.