Video bookmark based on soundtrack identification and two-stage search for interactive-television

This paper presents a video retrieval system (VRS) for Interactive-Television as like internet protocol television (IPTV). A video bookmark initiated by users is performed based on snippets of the background soundtrack corresponding to the ongoing program. Our VRS has two special aspects compared with previous bookmark systems. First, we adopt the robust audio fingerprint feature of long-term logarithmic modified DCT modulation coefficients (LMDCT-MC) for audio indexing and retrieval. Second, we propose and apply a two-stage search (TSS) algorithm for fast searching. In the first stage of TSS, candidate video segments are roughly determined with audio index bit vectors (IBV) and then the optimal video clip is obtained by fingerprint bit vectors (FBV). We evaluate the proposed system with a database of 100 TV programs including news, panel discussions, music shows, advertisements, and dramas. The experimental results show that our VRS achieve fast search, robustness to noise and high precision of retrieval. A search accuracy of 99.67% was accomplished.

[1]  Derek Hoiem,et al.  Computer vision for music identification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  George Tzanetakis,et al.  Musical genre classification of audio signals , 2002, IEEE Trans. Speech Audio Process..

[3]  Ton Kalker,et al.  A Highly Robust Audio Fingerprinting System , 2002, ISMIR.

[4]  Markus Cremer,et al.  Scalable robust audio fingerprinting using MPEG-7 content description , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[5]  Pedro Cano,et al.  A review of algorithms for audio fingerprinting , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[6]  Michael Fink,et al.  Social- and Interactive-Television Applications Based on Real-Time Ambient-Audio Identification , 2006 .

[7]  Xuan Zhu,et al.  A Robust Music Retrieval System , 2006 .

[8]  Ton Kalker,et al.  A Highly Robust Audio Fingerprinting System With an Efficient Search Strategy , 2003 .