Replay Detection and Multi-stream Synchronization in CS: GO Game Streams Using Content-Based Image Retrieval and Image Signature Matching

In GameStory: The 2019 Video Game Analytics Challenge, two main tasks are nominated to solve in the challenge, which are replay detection - multi-stream synchronization, and game story summarization. In this paper, we propose a data-driven based approach to solve the first task: replay detection - multi-stream synchronization. Our solution aims to determine the replays which lie between two logo-transitional endpoints and synchronize them with their sources by extracting frames from videos, then applying image processing and retrieval remedies. In detail, we use the Bag of Visual Words approach to detect the logo-transitional endpoints, which contains multiple replays in between, then employ an Image Signature Matching algorithm for multi-stream synchronization and replay boundaries refinement. The best configuration of our proposed solution manages to achieve the second-highest scores in all evaluation metrics of the challenge.

[1]  Jamie Woodcock,et al.  The impacts of live streaming and Twitch.tv on the video game industry , 2018, Media, Culture & Society.

[2]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[3]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[4]  Andrew Zisserman,et al.  Three things everyone should know to improve object retrieval , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Kevin Mekul.  Automated Replay Detection and Multi-Stream Synchronization , 2019, MediaEval.

[6]  Wolfgang Reisig.  Petri Nets: An Introduction , 1985, EATCS Monographs on Theoretical Computer Science.

[7]  Chng Eng Siong,et al.  Soccer replay detection using scene transition structure analysis , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[8]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[9]  Michael Riegler,et al.  GameStory Task at MediaEval 2018 , 2018, MediaEval.

[10]  Nathan Edge,et al.  Evolution of the Gaming Experience: Live Video Streaming and the Emergence of a New Web Community , 2013 .

[11]  Marshall W. Bern,et al.  An image signature for any kind of image , 2002, Proceedings. International Conference on Image Processing.

[12]  Alan F. Smeaton,et al.  Semantic Analysis of Field Sports Video using a Petri-Net of Audio-Visual Concepts , 2009, Comput. J..

[13]  Ali Javed,et al.  Replay and key-events detection for sports video summarization using confined elliptical local ternary patterns and extreme learning machine , 2019, Applied Intelligence.