Semantic concept mining in cricket videos for automated highlight generation

This paper presents a novel approach towards automated highlight generation of broadcast sports video sequences from its extracted events and semantic concepts. A sports video is hierarchically divided into temporal partitions namely, megaslots, slots, and semantic entities, namely concepts, and events. The proposed method extracts event sequence from video and classifies each sequence into a concept by sequential association mining. The extracted concepts and events within the concepts are selected according to their degree of importance to include those in the highlights. A parameter degree of abstraction is proposed, which gives a choice to the user about how concisely the extracted concepts should be produced for a specified highlight duration. We have successfully extracted highlights from recorded video of cricket match and compared our results with the manually-generated highlights by sports television channel.

[1]  Tat-Seng Chua,et al.  The fusion of audio-visual features and external knowledge for event detection in team sports video , 2004, MIR '04.

[2]  Somnath Sengupta,et al.  Semantic concept extraction from sports video for highlight generation , 2006, MobiMedia '06.

[3]  Masaharu Ogawa,et al.  A highlight scene detection and video summarization system using audio feature for a personal video recorder , 2005, IEEE Transactions on Consumer Electronics.

[4]  Alberto Del Bimbo,et al.  Taking into Consideration Sports Semantic Annotation of Sports Videos Content-based Multimedia Indexing and Retrieval , 2002 .

[5]  Sanjay N. Talbar,et al.  Texture Segmentation using Fractal Signature , 2000 .

[6]  S. Sengupta,et al.  Hidden Markov model based video indexing with discrete cosine transform as a likelihood function , 2004, Proceedings of the IEEE INDICON 2004. First India Annual Conference, 2004..

[7]  Qi Tian,et al.  A unified framework for semantic shot classification in sports video , 2005, IEEE Trans. Multim..

[8]  Riccardo Leonardi,et al.  Semantic indexing of soccer audio-visual sequences: a multimodal approach based on controlled Markov chains , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  C.-C. Jay Kuo,et al.  Rule-based video classification system for basketball video indexing , 2000, MULTIMEDIA '00.

[10]  Joemon M. Jose,et al.  Audio-Based Event Detection for Sports Video , 2003, CIVR.

[11]  Alberto Del Bimbo,et al.  Editorial: Introduction to the Special Issue on Multimedia Data Mining , 2008, IEEE Trans. Multim..

[12]  Regunathan Radhakrishnan,et al.  Audio events detection based highlights extraction from baseball, golf and soccer games in a unified framework , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[13]  Chiou-Ting Hsu,et al.  Fusion of audio and motion information on HMM-based highlight extraction for baseball games , 2006, IEEE Transactions on Multimedia.

[14]  Dragutin Petkovic,et al.  Content-based representation and retrieval of visual media: A state-of-the-art review , 1996, Multimedia Tools and Applications.

[15]  Alberto Del Bimbo,et al.  An Integrated Framework for Semantic Annotation and Adaptation , 2005, Multimedia Tools and Applications.

[16]  Baoxin Li,et al.  A general framework for sports video summarization with its application to soccer , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[17]  Lie Lu,et al.  Robust learning-based TV commercial detection , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[18]  Anoop Gupta,et al.  Automatically extracting highlights for TV Baseball programs , 2000, ACM Multimedia.

[19]  John R. Smith,et al.  On the detection of semantic concepts at TRECVID , 2004, MULTIMEDIA '04.

[20]  Mei Han,et al.  Extract highlights from baseball game video with hidden Markov models , 2002, Proceedings. International Conference on Image Processing.

[21]  Alan Hanjalic,et al.  Adaptive extraction of highlights from a sport video based on excitement modeling , 2005, IEEE Transactions on Multimedia.

[22]  A. Murat Tekalp,et al.  Automatic Soccer Video Analysis and Summarization , 2003, IS&T/SPIE Electronic Imaging.

[23]  Chng Eng Siong,et al.  Generation of Personalized Music Sports Video Using Multimodal Cues , 2007, IEEE Transactions on Multimedia.

[24]  Wen Gao,et al.  Human Behavior Analysis for Highlight Ranking in Broadcast Racket Sports Video , 2007, IEEE Transactions on Multimedia.

[25]  Xindong Wu,et al.  Video data mining: semantic indexing and event detection from the association perspective , 2005, IEEE Transactions on Knowledge and Data Engineering.

[26]  Tao Mei,et al.  Sports Video Mining with Mosaic , 2005, 11th International Multimedia Modelling Conference.

[27]  Noboru Babaguchi,et al.  Personalized abstraction of broadcasted American football video by highlight selection , 2004, IEEE Transactions on Multimedia.

[28]  Avideh Zakhor,et al.  Applications of Video-Content Analysis and Retrieval , 2002, IEEE Multim..

[29]  Ichiro Ide,et al.  An object detection method for describing soccer games from video , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[30]  R. Dahyot,et al.  Browsing sports video: trends in sports-related indexing and retrieval work , 2006, IEEE Signal Processing Magazine.

[31]  Zhu Liu,et al.  Joint scene classification and segmentation based on hidden Markov model , 2005, IEEE Transactions on Multimedia.

[32]  Noboru Babaguchi,et al.  Video Summarization for Large Sports Video Archives , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[33]  Shih-Fu Chang,et al.  Algorithms and system for segmentation and structure analysis in soccer video , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[34]  Takeo Kanade,et al.  Techniques for the Creation and Exploration of Digital Video Libraries , 1996 .

[35]  Changsheng Xu,et al.  Efficient multimodal features for automatic soccer highlight generation , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[36]  Mei Han,et al.  Baseball scene classification using multimedia features , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[37]  Somnath Sengupta,et al.  Event-Importance Based Customized and Automatic Cricket Highlight Generation , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[38]  Somnath Sengupta,et al.  A Hierarchical Framework for Generic Sports Video Classification , 2006, ACCV.

[39]  Baoxin Li,et al.  Semantic sports video analysis: approaches and new applications , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[40]  Lei Zhang,et al.  Canny edge detection enhancement by scale multiplication , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  C. V. Jawahar,et al.  Text Driven Temporal Segmentation of Cricket Videos , 2006, ICVGIP.

[42]  John M. Gauch,et al.  Identification of new commercials using repeated video sequence detection , 2005, IEEE International Conference on Image Processing 2005.

[43]  M.H. Kolekar,et al.  Semantic Indexing of News Video Sequences: A Multimodal Hierarchical Approach Based on Hidden Markov Model , 2005, TENCON 2005 - 2005 IEEE Region 10 Conference.

[44]  Patrick Gros,et al.  HMM based structuring of tennis videos using visual and audio cues , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[45]  Dragutin Petkovic,et al.  Content-Based Representation and Retrieval of Visual Media: A State-of-the-Art Review , 1996 .

[46]  Ajay Divakaran,et al.  Rapid generation of sports video highlights using the MPEG-7 motion activity descriptor , 2001, IS&T/SPIE Electronic Imaging.

[47]  Kayvan Najarian,et al.  Biomedical Signal and Image Processing , 2005 .