Content-based video indexing for sports applications using integrated multi-modal approach

To sustain an ongoing rapid growth of video information, there is an emerging demand for a sophisticated content-based video indexing system. However, current video indexing solutions are still immature and lack of any standard. This doctoral consists of a research work based on an integrated multi-modal approach for sports video indexing and retrieval. By combining specific features extractable from multiple audio-visual modalities, generic structure and specific events can be detected and classified. During browsing and retrieval, users will benefit from the integration of high-level semantic and some descriptive mid-level features such as whistle and close-up view of player(s).

[1]  Xinbo Gao,et al.  Speech retrieval with video parsing for television news programs , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[2]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[3]  Regunathan Radhakrishnan,et al.  Motion activity-based extraction of key-frames from video shots , 2002, Proceedings. International Conference on Image Processing.

[4]  Rainer Lienhart,et al.  On the segmentation of text in videos , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[5]  Hong-Jiang Zhang,et al.  Content-based video browsing and retrieval , 1999 .

[6]  Duane Szafron,et al.  Modeling video temporal relationships in an object database management system , 1997, Electronic Imaging.

[7]  Hao Jiang,et al.  Video segmentation with the assistance of audio content analysis , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[8]  Yeong-Ho Ha,et al.  Genre classification system of TV sound signals based on a spectrogram analysis , 1998 .

[9]  Fernando Pereira,et al.  MPEG-7 the generic multimedia content description standard, part 1 - Multimedia, IEEE , 2001 .

[10]  Tsuyoshi Moriyama,et al.  Video summarisation based on the psychological content in the track structure , 2000, MULTIMEDIA '00.

[11]  Marcel Worring,et al.  Multimodal Video Indexing : A Review of the State-ofthe-art , 2001 .

[12]  Qian Huang,et al.  Multimedia search and retrieval: new concepts, system implementation, and application , 2000, IEEE Trans. Circuits Syst. Video Technol..

[13]  Ellen K. Hughes,et al.  Video OCR for digital news archive , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[14]  A. Murat Tekalp,et al.  Integrated semantic-syntactic video event modeling for search and retrieval , 2002, Proceedings. International Conference on Image Processing.

[15]  Mohand-Said Hacid,et al.  A Database Approach for Modeling and Querying Video Data , 2000, IEEE Trans. Knowl. Data Eng..

[16]  Shih-Fu Chang,et al.  Algorithms and system for segmentation and structure analysis in soccer video , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[17]  Anil K. Jain,et al.  Face Detection in Color Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Hisashi Miyamori,et al.  Video annotation for content-based retrieval using human behavior analysis and domain knowledge , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[19]  Minoru Etoh,et al.  Rapid generation of event-based indexes for personalized video digests , 2002, Object recognition supported by user interaction for service robots.

[20]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[21]  Katsumi Tanaka,et al.  Video Database Systems - Recent Trends in Research and Development Activities , 1997, Handbook of Multimedia Information Management.

[22]  B. Li,et al.  Event detection and summarization in sports video , 2001, Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL 2001).

[23]  Chabane Djeraba Content-based multimedia indexing and retrieval , 2002, IEEE MultiMedia.

[24]  Tanveer F. Syeda-Mahmood,et al.  Learning video browsing behavior and its application in the generation of video previews , 2001, MULTIMEDIA '01.

[25]  David J. Crandall,et al.  Robust detection of stylized text events in digital video , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[26]  Richard M. Schwartz,et al.  Videotext OCR using hidden Markov models , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[27]  Wei-Ying Ma,et al.  Video summarization based on user log enhanced link analysis , 2003, ACM Multimedia.

[28]  Chuan Wu,et al.  Events recognition by semantic inference for sports video , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[29]  Yongmin Kim,et al.  Video object tracking with a sequential hierarchy of template deformations , 2001, IEEE Trans. Circuits Syst. Video Technol..

[30]  V. S. Subrahmanian,et al.  The CPR model for summarizing video , 2003, MMDB '03.

[31]  Shih-Fu Chang,et al.  Structure analysis of sports video using domain models , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[32]  C.-C. Jay Kuo,et al.  Audio content analysis for online audiovisual data segmentation and classification , 2001, IEEE Trans. Speech Audio Process..

[33]  Wei Jyh Heng Shot boundary refinement for long transition in digital video sequence , 2002, IEEE Trans. Multim..

[34]  Peter J. L. van Beek,et al.  Detection of slow-motion replay segments in sports video for highlights generation , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[35]  Riccardo Leonardi,et al.  Semantic Indexing of Multimedia Documents , 2002, IEEE Multim..

[36]  Bo Zhang,et al.  An efficient and effective region-based image retrieval framework , 2004, IEEE Trans. Image Process..

[37]  Katsumi Tanaka,et al.  Querying Video Data by Spatio-Temporal Relationships of Moving Object Traces , 2002, VDB.

[38]  Alan Hanjalic,et al.  Shot-boundary detection: unraveled and resolved? , 2002, IEEE Trans. Circuits Syst. Video Technol..

[39]  David S. Doermann,et al.  Detection of slow-motion replay sequences for identifying sports videos , 1999, 1999 IEEE Third Workshop on Multimedia Signal Processing (Cat. No.99TH8451).

[40]  Hang Joon Kim,et al.  Support vector machine-based text detection in digital video , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[41]  Ramesh C. Jain,et al.  Video Data Management Systems: Metadata and Architecture , 1998, Multimedia Data Management.

[42]  Takeo Kanade,et al.  Name-It: Naming and Detecting Faces in News Videos , 1999, IEEE Multim..

[43]  Mohamed Abdel-Mottaleb,et al.  Multimedia descriptions based on MPEG-7: extraction and applications , 2004, IEEE Transactions on Multimedia.

[44]  Rainer Lienhart,et al.  Automatic text recognition for video indexing , 1997, MULTIMEDIA '96.

[45]  Anil K. Jain,et al.  Automatic classification of tennis video for high-level content-based retrieval , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[46]  David J. Crandall,et al.  Robust extraction of text in video , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[47]  Anil C. Kokaram,et al.  A new global motion estimation algorithm and its application to retrieval in sports events , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[48]  Kien A. Hua,et al.  Semantics Reasoning Based Video Database Systems , 2000, DEXA.

[49]  Xuejing Sun,et al.  Pitch determination and voice quality analysis using Subharmonic-to-Harmonic Ratio , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[50]  Anoop Gupta,et al.  Automatically extracting highlights for TV Baseball programs , 2000, ACM Multimedia.

[51]  Sethuraman Panchanathan,et al.  A critical evaluation of image and video indexing techniques in the compressed domain , 1999, Image Vis. Comput..

[52]  Zhu Liu,et al.  Multimedia content analysis-using both audio and visual clues , 2000, IEEE Signal Process. Mag..

[53]  Michael J. Witbrock,et al.  Story segmentation and detection of commercials in broadcast news video , 1998, Proceedings IEEE International Forum on Research and Technology Advances in Digital Libraries -ADL'98-.

[54]  Noboru Babaguchi,et al.  Linking live and replay scenes in broadcasted sports video , 2000, MULTIMEDIA '00.

[55]  Qi Tian,et al.  Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video , 2003, MULTIMEDIA '03.

[56]  K. Selçuk Candan,et al.  The Advanced Video Information System: data structures and query processing , 1996, Multimedia Systems.

[57]  Willem Jonker,et al.  A Framework for Video Modeling , 2000 .

[58]  A. Murat Tekalp,et al.  Generic play-break event detection for summarization and hierarchical sports video analysis , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[59]  John R. Kender,et al.  Video scene segmentation via continuous video coherence , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[60]  Tat-Seng Chua,et al.  The Segmentation and Classification of Story Boundaries in News Video , 2002, VDB.

[61]  Takeo Kanade,et al.  Semantic analysis for video contents extraction—spotting by association in news video , 1997, MULTIMEDIA '97.

[62]  Vojkan Mihajlovic,et al.  Automatic Annotation of Formula 1 Races for Content-Based Video Retrieval , 2001 .

[63]  Silvia Pfeiffer,et al.  Pause concepts for audio segmentation at different semantic levels , 2001, MULTIMEDIA '01.

[64]  Tat-Seng Chua,et al.  Retrieval of News Video Using Video Sequence Matching , 2005, 11th International Multimedia Modelling Conference.

[65]  Yihong Gong,et al.  Feature design in soccer video indexing , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[66]  A. Murat Tekalp,et al.  Automatic Soccer Video Analysis and Summarization , 2003, IS&T/SPIE Electronic Imaging.

[67]  Serhan Dagtas,et al.  Extraction of TV highlights using multimedia features , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[68]  C.-C. Jay Kuo,et al.  Rule-based video classification system for basketball video indexing , 2000, MULTIMEDIA '00.

[69]  Lie Lu,et al.  A robust audio classification and segmentation method , 2001, MULTIMEDIA '01.

[70]  Harald Kosch,et al.  VIDEX: an integrated generic video indexing approach , 2000, MM 2000.

[71]  Yukinobu Taniguchi,et al.  PanoramaExcerpts: extracting and packing panoramas for video browsing , 1997, MULTIMEDIA '97.

[72]  Nilesh V. Patel,et al.  Video Segmentation for Video Data Management , 1997, Handbook of Multimedia Information Management.

[73]  Takeshi Mita,et al.  Improvement of video text recognition by character selection , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[74]  P. Beek,et al.  Text of 15938-5 FCD Information Technology-Multimedia Content Description Interface-Pard 5 Multimedia Description Schemes , 2001 .

[75]  Qi Tian,et al.  A mid-level representation framework for semantic sports video analysis , 2003, ACM Multimedia.

[76]  HongJiang Zhang,et al.  Video Snapshot: A Bird View of Video Sequence , 2005, 11th International Multimedia Modelling Conference.

[77]  Shih-Fu Chang,et al.  Overview of the MPEG-7 standard , 2001, IEEE Trans. Circuits Syst. Video Technol..

[78]  Rainer Lienhart,et al.  Localizing and segmenting text in images and videos , 2002, IEEE Trans. Circuits Syst. Video Technol..

[79]  Jenq-Neng Hwang,et al.  Fast and automatic video object segmentation and tracking for content-based applications , 2002, IEEE Trans. Circuits Syst. Video Technol..

[80]  Jun Yu,et al.  An efficient method for scene cut detection , 2001, Pattern Recognit. Lett..

[81]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[82]  NackFrank,et al.  Everything You Wanted to Know About MPEG-7 , 1999 .

[83]  Hang Joon Kim,et al.  A video indexing system using character recognition , 2000 .

[84]  Seong-Whan Lee,et al.  Automatic video parsing using shot boundary detection and camera operation analysis , 2001, Pattern Recognit..

[85]  HongJiang Zhang,et al.  Automatic video scene extraction by shot grouping , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[86]  Baoxin Li,et al.  Automatic detection of replay segments in broadcast sports programs by detection of logos in scene transitions , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[87]  Noboru Babaguchi,et al.  Event based indexing of broadcasted sports video by intermodal collaboration , 2002, IEEE Trans. Multim..

[88]  Thomas D. C. Little,et al.  Data Semantics for Improving Retrieval Performance of Digital News Video Systems , 1999, IEEE Trans. Knowl. Data Eng..

[89]  Carolyn E. Begg,et al.  Database Systems: A Practical Approach to Design, Implementation and Management , 1998 .

[90]  HongJiang Zhang,et al.  Automatic parsing of TV soccer programs , 1995, Proceedings of the International Conference on Multimedia Computing and Systems.

[91]  Lihi Zelnik-Manor,et al.  Event-based analysis of video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[92]  Surya Nepal,et al.  Automatic detection of 'Goal' segments in basketball videos , 2001, MULTIMEDIA '01.

[93]  Shih-Fu Chang,et al.  The holy grail of content-based media analysis , 2002 .

[94]  C.-C. Jay Kuo,et al.  On-line knowledge- and rule-based video classification system for video indexing and dissemination , 2002, Inf. Syst..

[95]  Arbee L. P. Chen,et al.  Semantic video model for content-based retrieval , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[96]  Alin Deutsch,et al.  XML-QL: A Query Language for XML , 1998 .

[97]  Shih-Fu Chang,et al.  Structure analysis of soccer video with hidden Markov models , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[98]  Kien A. Hua,et al.  Efficient and cost-effective techniques for browsing and indexing large video databases , 2000, SIGMOD 2000.

[99]  Nevenka Dimitrova,et al.  Text detection for video analysis , 1999, Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL'99).

[100]  Milind R. Naphade,et al.  Video retrieval and relevance feedback in the context of a post-integration model , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[101]  Wei-Pang Yang,et al.  A New Content-Based Access Method for Video Databases , 1999, Inf. Sci..

[102]  Yi Zhang,et al.  Detection of text captions in compressed domain video , 2000, MULTIMEDIA '00.