Hidden conditional random field-based soccer video events detection

Detect highlight event is an important step for semantic-based video retrieval. Hidden conditional random field (HCRF) is a discriminative model, which is effective in fusing observations for event inference. Mid-level semantics and their refinements are more robust than low-level visual features in event detection for learning models. To make full use of the contextual information, two aspects are taken into account during soccer video event detection. The first is parsing video sequences into event clips. The second is fusing the temporal transitions of the mid-level semantics of an event clip to determine the event type. In this study, HCRF is utilised to model the observations of mid-level semantics of an event clip for event detection. Comparisons are made with the dynamic Bayesian networks, hidden Markov model (HMM), enhanced HMM and conditional random field-based event detection approaches. Experimental results show the effectiveness of the proposed method.

[1]  Daniel Jurafsky,et al.  Regularization, adaptation, and non-independent features improve hidden conditional random fields for phone classification , 2007, 2007 IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU).

[2]  Andrew McCallum,et al.  Efficiently Inducing Features of Conditional Random Fields , 2002, UAI.

[3]  Noboru Babaguchi,et al.  Mining temporal information and web-casting text for automatic sports event detection , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[4]  Alex Acero,et al.  Training Algorithms for Hidden Conditional Random Fields , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[5]  Trevor Darrell,et al.  Hidden Conditional Random Fields , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Alex Acero,et al.  Hidden conditional random fields for phone classification , 2005, INTERSPEECH.

[7]  A. Murat Tekalp,et al.  Generic play-break event detection for summarization and hierarchical sports video analysis , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[8]  Qingming Huang,et al.  Highlight Summarization in Sports Video Based on Replay Detection , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[9]  Yi Ding,et al.  Two-Layer Generative Models for Sport Video Mining , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[10]  Fernando Pereira,et al.  Shallow Parsing with Conditional Random Fields , 2003, NAACL.

[11]  Liang-Tien Chia,et al.  Semantic Analysis of Basketball Video Using Motion Information , 2004, PCM.

[12]  Noel E. O'Connor,et al.  Event detection in field sports video using audio-visual features and a support vector Machine , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Andrew McCallum,et al.  An Introduction to Conditional Random Fields for Relational Learning , 2007 .

[14]  Jintao Li,et al.  Dynamic Bayesian network based event detection for soccer highlight extraction , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[15]  Xueming Qian,et al.  Visual summarization of landmarks via viewpoint modeling , 2012, 2012 19th IEEE International Conference on Image Processing.

[16]  Shih-Fu Chang,et al.  Algorithms and system for segmentation and structure analysis in soccer video , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[17]  Chung-Lin Huang,et al.  Semantic analysis of soccer video using dynamic Bayesian network , 2006, IEEE Transactions on Multimedia.

[18]  Michael G. Strintzis,et al.  Accumulated motion energy fields estimation and representation for semantic event detection , 2008, CIVR '08.

[19]  Xueming Qian,et al.  Ball and Field Line Detection for Placed Kick Refinement , 2009, 2009 WRI Global Congress on Intelligent Systems.

[20]  Qi Tian,et al.  A mid-level representation framework for semantic sports video analysis , 2003, ACM Multimedia.

[21]  Min Chen,et al.  Neural network based framework for goal event detection in soccer videos , 2005, Seventh IEEE International Symposium on Multimedia (ISM'05).

[22]  A. Murat Tekalp,et al.  Automatic soccer video analysis and summarization , 2003, IEEE Trans. Image Process..

[23]  Baoxin Li,et al.  Automatic detection of replay segments in broadcast sports programs by detection of logos in scene transitions , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[24]  Alberto Del Bimbo,et al.  Semantic annotation of soccer videos: automatic highlights identification , 2003, Comput. Vis. Image Underst..

[25]  Marcel Worring,et al.  Multimedia event-based video indexing using time intervals , 2005, IEEE Transactions on Multimedia.

[26]  Min Chen,et al.  Exciting Event Detection Using Multi-level Multimodal Descriptors and Data Classification , 2006, Eighth IEEE International Symposium on Multimedia (ISM'06).

[27]  Yi-Ping Phoebe Chen,et al.  Classification of self-consumable highlights for soccer video summaries , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[28]  Baoxin Li,et al.  Bridging the semantic gap in sports video retrieval and summarization , 2004, J. Vis. Commun. Image Represent..

[29]  Xiao-Ping Zhang,et al.  ICA mixture hidden conditional random field model for sports event classification , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[30]  Qi Tian,et al.  Semantic Shot Classification in Sports Video , 2003, IS&T/SPIE Electronic Imaging.

[31]  Chiou-Ting Hsu,et al.  Fusion of audio and motion information on HMM-based highlight extraction for baseball games , 2006, IEEE Transactions on Multimedia.

[32]  Chang-Hsing Lee,et al.  Scene-based event detection for baseball videos , 2007, J. Vis. Commun. Image Represent..

[33]  Xueming Qian,et al.  Object Categorization Using Hierarchical Wavelet Packet Texture Descriptors , 2009, 2009 11th IEEE International Symposium on Multimedia.

[34]  Xindong Wu,et al.  Video data mining: semantic indexing and event detection from the association perspective , 2005, IEEE Transactions on Knowledge and Data Engineering.

[35]  Changsheng Xu,et al.  Using Webcast Text for Semantic Event Detection in Broadcast Sports Video , 2008, IEEE Transactions on Multimedia.

[36]  Shih-Fu Chang,et al.  Structure analysis of soccer video with hidden Markov models , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[37]  Tao Wang,et al.  Semantic Event Detection using Conditional Random Fields , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[38]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[39]  Loong Fah Cheong,et al.  Dynamic Bayesian framework for extracting temporal structure in video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[40]  Jianping Fan,et al.  ClassView: hierarchical video shot classification, indexing, and accessing , 2004, IEEE Transactions on Multimedia.

[41]  Rob Malouf,et al.  A Comparison of Algorithms for Maximum Entropy Parameter Estimation , 2002, CoNLL.

[42]  Xueming Qian,et al.  Global motion estimation from randomly selected motion vector groups and GM/LM based applications , 2007, Signal Image Video Process..

[43]  Shiqiang Yang,et al.  An HMM-based framework for video semantic analysis , 2005, IEEE Trans. Circuits Syst. Video Technol..

[44]  Chen Wang,et al.  An SVM-Based Soccer Video Shot Classification Scheme Using Projection Histograms , 2008, PCM.

[45]  Xueming Qian,et al.  HMM based soccer video event detection using enhanced mid-level semantic , 2011, Multimedia Tools and Applications.

[46]  Ming-Ting Sun,et al.  Global motion estimation from coarsely sampled motion vector field and the applications , 2005, IEEE Trans. Circuits Syst. Video Technol..

[47]  Jintao Li,et al.  A Generic Framework for Semantic Sports Video Analysis Using Dynamic Bayesian Networks , 2005, 11th International Multimedia Modelling Conference.

[48]  Xueming Qian,et al.  Highlight events detection in soccer video using HCRF , 2010, ICIMCS '10.

[49]  Regunathan Radhakrishnan,et al.  Highlights extraction from sports video based on an audio-visual marker detection framework , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[50]  Linmi Tao,et al.  Hidden Markov Model Based Events Detection in Soccer Video , 2004, ICIAR.

[51]  Peter J. L. van Beek,et al.  Detection of slow-motion replay segments in sports video for highlights generation , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[52]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[53]  Sanjeev R. Kulkarni,et al.  Rapid estimation of camera motion from compressed video with application to video annotation , 2000, IEEE Trans. Circuits Syst. Video Technol..

[54]  Noboru Babaguchi,et al.  Sports event detection using temporal patterns mining and web-casting text , 2008, AREA '08.

[55]  Qi Tian,et al.  A unified framework for semantic shot classification in sports video , 2005, IEEE Trans. Multim..

[56]  Shih-Fu Chang,et al.  Event detection in baseball video using superimposed caption recognition , 2002, MULTIMEDIA '02.

[57]  Changsheng Xu,et al.  A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video , 2008, IEEE Transactions on Multimedia.