HMM-based ball hitting event exploration system for broadcast baseball video

With the dramatic growth of fandom population, a considerable amount of research efforts have been devoted to baseball video processing. However, little work focuses on the detailed follow-ups of ball hitting events. This paper proposes a HMM-based ball hitting event exploration system for broadcast baseball video. Utilizing the strictly-defined layout of the baseball field, the proposed system first detects the game-specific spatial patterns in the field, such as the field lines, the bases, the pitch mound, etc. Then, the play region-the currently camera-focused region of the baseball field is identified for frame type classification. Since the temporal patterns of presenting the game progress follow a prototypical order, we consider the classified frame types as observation symbols and recognize ball hitting events using HMM. Experiments conducted on broadcast baseball video show encouraging results in frame type classification and ball hitting event recognition. Three practical applications, including highlight clip extraction by user-designated query, storyboard construction, and similar event retrieval, are introduced to address the applicability of our system.

[1]  Peter H. N. de With,et al.  Fast camera calibration for the analysis of sport sequences , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[2]  Hubert P. H. Shum,et al.  Tracking the translational and rotational movement of the ball using high-speed camera movies , 2005, IEEE International Conference on Image Processing 2005.

[3]  Bernd Jähne,et al.  BOOK REVIEW: Digital Image Processing, 5th revised and extended edition , 2002 .

[4]  Mei Han,et al.  Extract highlights from baseball game video with hidden Markov models , 2002, Proceedings. International Conference on Image Processing.

[5]  Mei Han,et al.  Maximum entropy model-based baseball highlight detection and classification , 2004, Comput. Vis. Image Underst..

[6]  Chang-Hsing Lee,et al.  Scene-based event detection for baseball videos , 2007, J. Vis. Commun. Image Represent..

[7]  Wen Gao,et al.  Event Tactic Analysis Based on Broadcast Sports Video , 2009, IEEE Trans. Multim..

[8]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[9]  Mao-Hsiung Hung,et al.  Event Detection of Broadcast Baseball Videos , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Chng Eng Siong,et al.  Automatic composition of broadcast sports video , 2008, Multimedia Systems.

[11]  Chien-Li Chou,et al.  Extraction and representation of human body for pitching style recognition in broadcast baseball video , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[12]  Min Xu,et al.  Multimodal Semantic Analysis and Annotation for Basketball Video , 2006, EURASIP J. Adv. Signal Process..

[14]  Nobuyuki Yagi,et al.  Baseball video indexing using patternization of scenes and hidden Markov model , 2005, IEEE International Conference on Image Processing 2005.

[15]  Hua-Tsung Chen,et al.  Physics-Based Ball Tracking in Volleyball Videos with its Applications to Set Type Recognition and Action Detection , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[16]  André Guéziec.  Tracking Pitches for Broadcast Television , 2002, Computer.

[17]  Patrick Gros,et al.  Audiovisual integration for tennis broadcast structuring , 2006, Multimedia Tools and Applications.

[18]  Hua-Tsung Chen,et al.  Contour-based strike zone shaping and visualization in broadcast baseball video: providing reference for pitch location positioning and strike/ball judgment , 2009, Multimedia Tools and Applications.

[19]  Peter N. Yianilos,et al.  Learning String-Edit Distance , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[21]  Alan Hanjalic,et al.  Shot-boundary detection: unraveled and resolved? , 2002, IEEE Trans. Circuits Syst. Video Technol..

[22]  Wei-Ta Chu,et al.  Explicit semantic events detection and development of realistic applications for broadcasting baseball videos , 2008, Multimedia Tools and Applications.

[23]  Wolfgang Effelsberg,et al.  Robust camera calibration for sport videos using court models , 2003, IS&T/SPIE Electronic Imaging.

[24]  Hua-Tsung Chen,et al.  A Trajectory-Based Ball Tracking Framework with Visual Enrichment for Broadcast Baseball Videos , 2008, J. Inf. Sci. Eng..

[25]  Chiou-Ting Hsu,et al.  Fusion of audio and motion information on HMM-based highlight extraction for baseball games , 2006, IEEE Transactions on Multimedia.

[26]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[27]  Min-Chun Hu,et al.  Robust Camera Calibration and Player Tracking in Broadcast Basketball Video , 2011, IEEE Transactions on Multimedia.

[28]  Loong Fah Cheong,et al.  Automatic camera calibration of broadcast tennis video with applications to 3D virtual content insertion and ball detection and tracking , 2009, Comput. Vis. Image Underst..

[29]  Qi Tian,et al.  Trajectory-Based Ball Detection and Tracking in Broadcast Soccer Video , 2006, IEEE Transactions on Multimedia.

[30]  Deb Roy,et al.  Temporal feature induction for baseball highlight classification , 2007, ACM Multimedia.

[31]  Hua-Tsung Chen,et al.  Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video , 2009, J. Vis. Commun. Image Represent..

[32]  Randal C. Nelson,et al.  Finding Line Segments by Stick Growing , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Qi Tian,et al.  A unified framework for semantic shot classification in sports video , 2005, IEEE Trans. Multim..

[34]  Yasuo Ariki,et al.  Automatic extraction of PC scenes based on feature mining for a real time delivery system of baseball highlight scenes , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).