Structure and event mining in sports video with efficient mosaic

Video is an information-intensive media with much redundancy. Therefore, it is desirable to be able to mine structure or semantics of video data for efficient browsing, summarization and highlight extraction. In this paper, we propose a mosaic based approach to key-event as well as structure mining, which is regarded as a complementary view for sports video analysis. Mosaic is generated for each shot by a novel efficient mosaicing scheme, which constructs a global motion path and selects a best subset of frames for mosaicing. These improved mosaics are then used as the representative image of shot content. Based on mosaic, the structure and event in sports video are mined by the methods with prior knowledge and without prior knowledge. Without prior knowledge, our system is able to locate global view shots taken by dominant camera. If prior knowledge is available, the events in these global view shots are detected using robust features extracted from mosaics. For global view mining, the experiments compared with key-frame-based scheme have demonstrated that this mosaic-based scheme presents better results in several kinds of sports videos; for events mining, the detection of key-plays and key-events in the specific-domain of soccer videos have proved its effectiveness.

[1]  Alan Hanjalic,et al.  Adaptive extraction of highlights from a sport video based on excitement modeling , 2005, IEEE Transactions on Multimedia.

[2]  Patrick Bouthemy,et al.  A unified approach to shot change detection and camera motion characterization , 1999, IEEE Trans. Circuits Syst. Video Technol..

[3]  A. Murat Tekalp,et al.  Generic play-break event detection for summarization and hierarchical sports video analysis , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[4]  Shih-Fu Chang,et al.  Structure analysis of soccer video with domain knowledge and hidden Markov models , 2004, Pattern Recognit. Lett..

[5]  Qi Tian,et al.  A unified framework for semantic shot representation of sports video , 2005, MIR '05.

[6]  Changsheng Xu,et al.  Live sports event detection based on broadcast video and web-casting text , 2006, MM '06.

[7]  Atreyi Kankanhalli,et al.  Automatic partitioning of full-motion video , 1993, Multimedia Systems.

[8]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[9]  Xinguo Yu,et al.  Trajectory-Based Ball Detection and Tracking in Broadcast Soccer Video with the Aid of Camera Motion Recovery , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[10]  Shih-Fu Chang,et al.  Structure analysis of sports video using domain models , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[11]  M. Luo,et al.  Pyramidwise structuring for soccer highlight extraction , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[12]  John R. Kender,et al.  Video Summaries through Mosaic-Based Shot and Scene Clustering , 2002, ECCV.

[13]  Yi Ding,et al.  Two-Layer Generative Models for Sport Video Mining , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[14]  Yongduek Seo,et al.  Where Are the Ball and Players? Soccer Game Analysis with Color Based Tracking and Image Mosaick , 1997, ICIAP.

[15]  Qi Tian,et al.  A unified framework for semantic shot classification in sports video , 2005, IEEE Trans. Multim..

[16]  Yi-Ping Phoebe Chen,et al.  The power of play-break for automatic detection and browsing of self-consumable sport video highlights , 2004, MIR '04.

[17]  Mei Han,et al.  Baseball scene classification using multimedia features , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[18]  Shih-Fu Chang,et al.  Algorithms and system for segmentation and structure analysis in soccer video , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[19]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[20]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[21]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[22]  Tao Wang,et al.  Soccer Highlight Detection using Two-Dependence Bayesian Network , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[23]  Tao Mei,et al.  Efficient video mosaicing based on motion analysis , 2005, IEEE International Conference on Image Processing 2005.

[24]  Martin R. Varley,et al.  Improved Video Mosaic Construction by Accumulated Alignment Error Distribution , 1998, BMVC.

[25]  Matthew A. Brown,et al.  Recognising panoramas , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[26]  Lei Zhang,et al.  A CBIR method based on color-spatial feature , 1999, Proceedings of IEEE. IEEE Region 10 Conference. TENCON 99. 'Multimedia Technology for Asia-Pacific Information Infrastructure' (Cat. No.99CH37030).

[27]  Chng Eng Siong,et al.  Identify Sports Video Shots with "Happy" or "Sad" Emotions , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[28]  Biswajit Bose,et al.  From video sequences to motion panoramas , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[29]  Alberto Del Bimbo,et al.  Soccer highlights detection and recognition using HMMs , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[30]  Richard Szeliski,et al.  Video mosaics for virtual environments , 1996, IEEE Computer Graphics and Applications.

[31]  Tao Mei,et al.  Sports Video Mining with Mosaic , 2005, 11th International Multimedia Modelling Conference.