Automatic Live Sport Video Streams Curation System from User Generated Media

Emerging Internet of Things IoT technologies will allow spectators in a sport game to produce various video streams from various angles. With existing technologies, however, it is difficult to process massive and various data streams for multi-channel contents in real-time. To solve this problem, we aim to construct a software agent called "Curator" that compiles video contents automatically according to his/her values. In this paper, we propose a system to automatically switch multiple video streams that general sports spectators have taken using Random Forests classifier. Meta data such as image feature data and game progress data is extracted for each video scene as the input of the classifier. For evaluation, we constructed a camera switching timing estimation model using the live TV broadcast of some baseball game data. A video of another baseball game was curated with the constructed model. As a result, our system predicted the camera switching timing with accuracy F-measure of 85.3% on weighted average for the base camera work and 99.7% for the fixed camera work.

[1]  Abhishek Ranjan,et al.  Improving meeting capture by applying television production principles with audio and motion detection , 2008, CHI.

[2]  Jeffrey Nichols,et al.  Summarizing sporting events using twitter , 2012, IUI '12.

[3]  Takeo Igarashi,et al.  AnnoTone: Record-time Audio Watermarking for Context-aware Video Editing , 2015, CHI.

[4]  Changsheng Xu,et al.  A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video , 2008, IEEE Transactions on Multimedia.

[5]  Abderrahim Sekkaki,et al.  Soccer Video Summarization Using Video Content Analysis and Social Media Streams , 2014, 2014 IEEE/ACM International Symposium on Big Data Computing.

[6]  Hirozumi Yamaguchi,et al.  Survey of Real-time Processing Technologies of IoT Data Streams , 2016, J. Inf. Process..

[7]  Winston H. Hsu,et al.  Live Semantic Sport Highlight Detection Based on Analyzing Tweets of Twitter , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[8]  David Conejero,et al.  Audio based soccer game summarization , 2009, 2009 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting.

[9]  Rene Kaiser,et al.  Experimental Enquiry into Automatically Orchestrated Live Video Communication in Social Settings , 2015, TVX.

[10]  Jason J. Jung,et al.  Real-time Event Detection on Social Data Stream , 2014, Mobile Networks and Applications.

[11]  Marc Davis,et al.  Metadata creation system for mobile images , 2004, MobiSys '04.

[12]  Anoop Gupta,et al.  Automatically extracting highlights for TV Baseball programs , 2000, ACM Multimedia.

[13]  Osamu Nakamura,et al.  SENSeTREAM: enhancing online live experience with sensor-federated video stream using animated two-dimensional code , 2014, UbiComp.

[14]  James J. Little,et al.  Identifying players in broadcast sports videos using conditional random fields , 2011, CVPR 2011.

[15]  A. Murat Tekalp,et al.  Automatic soccer video analysis and summarization , 2003, IEEE Trans. Image Process..

[16]  Takuro Yonezawa,et al.  Enhancing communication and dramatic impact of online live performance with cooperative audience control , 2012, UbiComp '12.

[17]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[18]  Benjamin Wulff,et al.  LectureSight - An Open Source System for Automatic Camera Control in Lecture Recordings , 2012, 2012 IEEE International Symposium on Multimedia.

[19]  Gregory D. Abowd,et al.  Automatic Synchronization of Wearable Sensors and Video-Cameras for Ground Truth Annotation -- A Practical Approach , 2012, 2012 16th International Symposium on Wearable Computers.

[20]  Maria da Graça Campos Pimentel,et al.  Multimedia multi-device educational presentations preserved as interactive multi-video objects , 2013, WebMedia.

[21]  Giovanni Maria Farinella,et al.  RECfusion: Automatic Video Curation Driven by Visual Content Popularity , 2015, ACM Multimedia.

[22]  Takatsugu Hirayama,et al.  Context-Dependent Viewpoint Sequence Recommendation System for Multi-view Video , 2014, 2014 IEEE International Symposium on Multimedia.

[23]  Shihong Lao,et al.  Multiple Player Tracking in Sports Video: A Dual-Mode Two-Way Bayesian Inference Approach With Progressive Observation Modeling , 2011, IEEE Transactions on Image Processing.