A quick search method for audio and video signals based on histogram pruning

This paper proposes a quick method of similarity-based signal searching to detect and locate a specific audio or video signal given as a query in a stored long audio or video signal. With existing techniques, similarity-based searching may become impractical in terms of computing time in the case of searching through long-running (several-days' worth of) signals. The proposed algorithm, which is referred to as time-series active search, offers significantly faster search with sufficient accuracy. The key to the acceleration is an effective pruning algorithm introduced in the histogram matching stage. Through the pruning, the actual number of matching calculations can be reduced by 200 to 500 times compared with exhaustive search while guaranteeing exactly the same search result. Experiments show that the proposed method can correctly detect and locate a 15-s signal in a 48-h recording of TV broadcasts within 1 s, once the feature vectors are calculated and quantized. As extentions of the basic algorithm, efficient AND/OR search methods for searching for multiple query signals and a feature dithering method for coping with signal distortion are also discussed.

[1]  John C. Hancock,et al.  Signal Detection Theory , 1966 .

[2]  M. R. Schroeder,et al.  Adaptive predictive coding of speech signals , 1970, Bell Syst. Tech. J..

[3]  Robert S. Boyer,et al.  A fast string searching algorithm , 1977, CACM.

[4]  Donald E. Knuth,et al.  Fast Pattern Matching in Strings , 1977, SIAM J. Comput..

[5]  B. Kedem,et al.  Spectral analysis and discrimination by zero-crossings , 1986, Proceedings of the IEEE.

[6]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[7]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[8]  Douglas Keislar,et al.  Content-Based Classification, Search, and Retrieval of Audio , 1996, IEEE Multim..

[9]  Michael Shneier,et al.  Exploiting the JPEG Compression Scheme for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  John Saunders,et al.  Real-time discrimination of broadcast speech/music , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[11]  Wolfgang Effelsberg,et al.  Automatic audio content analysis , 1997, MULTIMEDIA '96.

[12]  Jonathan Foote,et al.  Content-based retrieval of music and audio , 1997, Other Conferences.

[13]  Karen Spärck Jones,et al.  Acoustic indexing for multimedia retrieval and browsing , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[14]  Hiroshi Murase,et al.  Focused color intersection with efficient searching for object extraction , 1997, Pattern Recognit..

[15]  Kunio Kashino,et al.  Quick audio retrieval using active search , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[16]  C.-C. Jay Kuo,et al.  Hierarchical system for content-based audio classification and retrieval , 1998, Other Conferences.

[17]  Kunio Kashino,et al.  Time-series active search for quick retrieval of audio and video , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[18]  Thomas S. Huang,et al.  Edge / Structural Features for Content Based Image Retrieval , 1999 .

[19]  Jonathan Foote,et al.  An overview of audio information retrieval , 1999, Multimedia Systems.

[20]  Beth Logan,et al.  Mel Frequency Cepstral Coefficients for Music Modeling , 2000, ISMIR.

[21]  Howard D. Wactlar,et al.  Informedia - Search and Summarization in the Video Medium , 2000 .

[22]  Roberto Brunelli,et al.  Image Retrieval by Examples , 2000, IEEE Trans. Multim..

[23]  Thomas S. Huang,et al.  Edge-based structural features for content-based image retrieval , 2001, Pattern Recognit. Lett..

[24]  Beth Logan,et al.  Speechbot: an experimental speech-based search engine for multimedia content on the web , 2002, IEEE Trans. Multim..

[25]  B. S. Manjunath,et al.  Texture-based pattern retrieval from image databases , 2004, Multimedia Tools and Applications.

[26]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[27]  Yihong Gong,et al.  Image indexing and retrieval based on color histograms , 1996, Multimedia Tools and Applications.

[28]  Hou Yi-bin.  A Fast String Matching Algorithm , 2004 .

[29]  Babu M. Mehtre,et al.  CORE: a content-based retrieval engine for multimedia information systems , 1995, Multimedia Systems.