Self-similarity-based partial near-duplicate video retrieval and alignment

There have been recent studies on partial near-duplicate videos, which involve segments of videos that are near duplicates of each other. State-of-the-art searching schemes usually segment the input video into clips and implement clip-level near-duplicate retrieval. However, the segmentation results are always poorly aligned, which lead to a difficult “unbalance” problem. In this paper, we introduce a self-similarity-based feature representation called the Self-Similarity Belt (SSBelt), which derives from the Self-Similarity Matrix (SSM). In addition, a distinctive pattern in SSBelt called the Interest Corner is detected and described by a bag-of-words representation. The visual words are then combined into visual shingles and indexed by an inverted file index for fast retrieval. Another important task is to accurately align the unbalanced clips, for which we propose the Intensity Mark (IMark) and design a coarse-to-fine near-duplicate video localization scheme. Experimental results show the effectiveness of our approach for both web-based near-duplicate video and unbalanced video datasets. The near-duplicate alignment capacity of IMark is also shown to be effective.