3D ball trajectory reconstruction from single-camera sports video for free viewpoint virtual replay

Free viewpoint video presentation is a new challenge in multimedia analysis. This paper presents an innovative physics-based scheme to reconstruct the 3D ball trajectory from single-camera volleyball video sequences for free viewpoint virtual replay. The problem of 2D-to-3D inference is arduous due to the loss of 3D information in projection to 2D images. The proposed scheme incorporates the domain knowledge of court specification and the physical characteristics of ball motion to accomplish the 2D-to-3D inference. Motion equations with the parameters are set up to define the 3D trajectories based on physical characteristics. Utilizing the geometric transformation of camera calibration, the 2D ball coordinates extracted over frames are used to approximate the parameters of the 3D motion equations, and finally the 3D ball trajectory can be reconstructed from single-camera sequences. The experiments show promising results. The reconstructed 3D trajectory enables the free viewpoint virtual replay and enriched visual presentation, making game watching a whole new experience.

[1]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[2]  Chiou-Ting Hsu,et al.  Fusion of audio and motion information on HMM-based highlight extraction for baseball games , 2006, IEEE Transactions on Multimedia.

[3]  Alberto Del Bimbo,et al.  Semantic annotation of soccer videos: automatic highlights identification , 2003, Comput. Vis. Image Underst..

[4]  Suh-Yin Lee,et al.  A baseball exploration system using spatial pattern recognition , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[5]  Hua-Tsung Chen,et al.  Physics-Based Ball Tracking in Volleyball Videos with its Applications to Set Type Recognition and Action Detection , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[6]  H. Opower.  Multiple view geometry in computer vision , 2002 .

[7]  Hua-Tsung Chen,et al.  Shot Classification of Basketball Videos and its Application in Shooting Position Extraction , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[8]  André Guéziec.  Tracking Pitches for Broadcast Television , 2002, Computer.

[9]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[10]  Chien-Li Chou,et al.  Extraction and representation of human body for pitching style recognition in broadcast baseball video , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[11]  N. Owens,et al.  Hawk-eye tennis system , 2003 .

[12]  Qi Tian,et al.  A unified framework for semantic shot classification in sports video , 2002, IEEE Transactions on Multimedia.

[13]  Hua-Tsung Chen,et al.  Pitch-by-Pitch Extraction from Single View Baseball Video Sequences , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[14]  Thomas Wiegand,et al.  3D Video and Free Viewpoint Video - Technologies, Applications and MPEG Standards , 2006, 2006 IEEE International Conference on Multimedia and Expo.