Personal Viewpoint Navigation Based on Object Trajectory Distribution for Multi-View Videos

Multi-camera videos with abundant information and high flexibility are useful in a wide range of applications, such as surveillance systems, web lectures, news broadcasting, concerts and sports viewing. Viewers can enjoy an enhanced viewing experience by choosing their own viewpoint through viewing interfaces. However, some viewers may feel annoyed by the need for continual manual viewpoint selection, especially when the number of selectable viewpoints is relatively large. In order to solve this issue, we propose an automatic viewpoint navigation method designed especially for sports. This method focuses on a viewer’s personal preference for viewpoint selection, instead of common and professional editing rules. We assume that different trajectory distributions of viewing objects cause a difference in the viewpoint selection according to personal preference. We learn the relationship between the viewer’s personal viewpoint-selection tendency and the spatio-temporal game context represented by the objects trajectories. We compare three methods based on Gaussian mixture model, SVM with a general histogram and SVM with a bag-of-words to seek the best learning scheme for this relationship. The performance of the proposed methods are evaluated by assessing the degree of similarity between the selected viewpoints and the viewers’ edited records. key words: multi-view video navigation, user preference, Gaussian mixture model

[1]  Takatsugu Hirayama,et al.  Personal Multi-view Viewpoint Recommendation based on Trajectory Distribution of the Viewing Target , 2016, ACM Multimedia.

[2]  Yasuo Ariki,et al.  Video editing support system based on video grammar and content analysis , 2002, Object recognition supported by user interaction for service robots.

[3]  Limin Wang,et al.  Computer Vision and Image Understanding Bag of Visual Words and Fusion Methods for Action Recognition: Comprehensive Study and Good Practice , 2022 .

[4]  Yuichi Ohta,et al.  Free viewpoint browsing of live soccer games , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[5]  Yoshiyasu Takefuji,et al.  Model-based Behavioral Causality Analysis of Handball with Delayed Transfer Entropy , 2015, BICA.

[6]  Andrea Cavallaro,et al.  Multi-camera Scheduling for Video Production , 2011, 2011 Conference for Visual Media Production.

[7]  Andrea Cavallaro,et al.  Content and task-based view selection from multiple video streams , 2009, Multimedia Tools and Applications.

[8]  Norihiro Hagita,et al.  Ubiquitous Experience Media , 2006, IEEE Multim..

[9]  Takatsugu Hirayama,et al.  Video generation method based on user's tendency of viewpoint selection for multi-view video contents , 2014, AH.

[10]  Kwanghoon Sohn,et al.  A Real-Time 3D Modeling System Using Multiple Stereo Cameras for Free-Viewpoint Video Generation , 2006, ICIAR.

[11]  Tiziana D'Orazio,et al.  A review of vision-based systems for soccer video analysis , 2010, Pattern Recognit..

[12]  Aljoscha Smolic,et al.  Computational sports broadcasting: Automated director assistance for live sports , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[13]  Wei Tsang Ooi,et al.  MoViMash: online mobile video mashup , 2012, ACM Multimedia.

[14]  Takashi Matsuyama,et al.  3D Video and Its Applications , 2012, Springer London.

[15]  Abhishek Ranjan,et al.  Automatic camera control using unobtrusive vision and audio tracking , 2010, Graphics Interface.

[16]  Anoop Gupta,et al.  Distributed meetings: a meeting capture and broadcasting system , 2002, MULTIMEDIA '02.

[17]  Ishfaq Ahmad Multi-View Video: Get Ready for Next-Generation Television , 2007, IEEE Distributed Systems Online.

[18]  Hiroshi Sankoh,et al.  Interactive music video application for smartphones based on free-viewpoint video and audio rendering , 2012, ACM Multimedia.

[19]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[20]  Kenji Mase,et al.  Semi-automatic Multiple Player Tracking of Soccer Games using Laser Range Finders , 2016, AH.

[21]  Adrian Hilton,et al.  A Comparative Study of Free-Viewpoint Video Techniques For sports events , 2006 .

[22]  Jiang Li,et al.  A real-time interactive multi-view video system , 2005, MULTIMEDIA '05.

[23]  Kenji Mase,et al.  Socially assisted multi-view video viewer , 2011, ICMI '11.

[24]  Takatsugu Hirayama,et al.  Viewpoint Sequence Recommendation Based on Contextual Information for Multiview Video , 2015, IEEE MultiMedia.

[25]  Takatsugu Hirayama,et al.  Analysis of Soccer Coach's Eye Gaze Behavior , 2013, 2013 2nd IAPR Asian Conference on Pattern Recognition.

[26]  Thomas Sikora,et al.  A standards-based, flexible, end-to-end multi-view video streaming architecture , 2007, Packet Video 2007.

[27]  Jintao Li,et al.  Hierarchical spatio-temporal context modeling for action recognition , 2009, CVPR.

[28]  Sidney S. Fels,et al.  A Multi-Camera Surveillance System that Estimates Quality-of-View Measurement , 2007, 2007 IEEE International Conference on Image Processing.

[29]  Takeo Kanade,et al.  An active camera system for acquiring multi-view video , 2002, Proceedings. International Conference on Image Processing.