Automatic semantic analysis of gameplay videos of ‘This War of Mine’

In recent years there has been an increase in published research on video games. However, few articles discuss semantic analysis of gameplay videos, which are available online on platforms such as YouTube or Twitch. A case-study was performed where a three-step approach is proposed to analyze a speci c scene in videos of the game `This War of Mine'. The rst step is to detect the location in the game at which the scene of interest takes place using a SVM with bag-of-visual-words histograms of SIFT features. Then convolutional neural networks are used to detect which scene takes place at that location and what choice the player makes during

[1]  Gareth Schott,et al.  Feedback-based gameplay metrics: measuring player experience via automatic visual analysis , 2012, IE '12.

[2]  Wen-Huang Cheng,et al.  Semantic-event based analysis and segmentation of wedding ceremony videos , 2007, MIR '07.

[3]  Ioannis Patras,et al.  Deep Multi-task Learning with Label Correlation Constraint for Video Concept Detection , 2016, ACM Multimedia.

[4]  Konstantin Mitgutsch,et al.  Purposeful by design?: a serious game design assessment framework , 2012, FDG.

[5]  Chung-Lin Huang,et al.  Semantic analysis of soccer video using dynamic Bayesian network , 2006, IEEE Transactions on Multimedia.

[6]  Shih-Fu Chang,et al.  Structure analysis of sports video using domain models , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[7]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[8]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[9]  Wei-Ta Chu,et al.  Event Detection and Highlight Detection of Broadcasted Game Videos , 2015, HCMC '15.

[10]  Frédéric Maire,et al.  A Framework for the Semi-Automatic Testing of Video Games , 2008, AIIDE.

[11]  Alessandro Canossa,et al.  Meaning in Gameplay: Filtering Variables, Defining Metrics, Extracting Features and Creating Models for Gameplay Analysis , 2013, Game Analytics, Maximizing the Value of Player Data.

[12]  Wei-Ta Chu,et al.  Badminton Video Analysis based on Spatiotemporal and Stroke Features , 2017, ICMR.

[13]  Bhabatosh Chanda,et al.  A heuristic algorithm for video scene detection using shot cluster sequence analysis , 2010, ICVGIP '10.

[14]  Staffan Björk,et al.  Formal analysis of gameplay , 2015 .

[15]  Tao Mei,et al.  Near-lossless semantic video summarization and its applications to video analysis , 2013, TOMCCAP.

[16]  Tao Mei,et al.  EMS: Energy Minimization Based Video Scene Segmentation , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[17]  Magy Seif El-Nasr,et al.  Analysis of level design 'push & pull' within 21 games , 2010, FDG.

[18]  Kerstin Radde-Antweiler.  Video Gaming, Let's Plays, and Religion: The Relevance of Researching gamevironments. , 2014 .

[19]  Kristine Jørgensen,et al.  Audio and Gameplay: An Analysis of PvP Battlegrounds in World of Warcraft , 2008, Game Stud..

[20]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[21]  R. Veltkamp,et al.  Analysing player decision-making of a moral dilemma through a computer vision analysis of Youtube gameplay videos , 2017 .

[22]  Boon-Lock Yeo,et al.  Segmentation of Video by Clustering and Graph Analysis , 1998, Comput. Vis. Image Underst..

[23]  Ioannis Patras,et al.  VideoAnalysis4ALL: An On-line Tool for the Automatic Fragmentation and Concept-based Annotation, and the Interactive Exploration of Videos , 2017, ICMR.

[24]  Bohyung Han,et al.  MarioQA: Answering Questions by Watching Gameplay Videos , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[25]  Peter H. N. de With,et al.  Automatic video-based human motion analyzer for consumer surveillance system , 2009, IEEE Transactions on Consumer Electronics.

[26]  Liang-Hua Chen,et al.  Movie scene segmentation using background information , 2008, Pattern Recognit..

[27]  Pierre Manneback,et al.  Semantic analysis of human movements in videos , 2012, I-SEMANTICS '12.

[28]  C.-C. Jay Kuo,et al.  Rule-based video classification system for basketball video indexing , 2000, MULTIMEDIA '00.

[29]  Bernd Freisleben,et al.  Semantic video analysis for psychological research on violence in computer games , 2007, CIVR '07.

[30]  Michael G. Strintzis,et al.  An Ontology Framework For Knowledge-Assisted Semantic Video Analysis and Annotation , 2004, SemAnnot@ISWC.

[31]  Qi Tian,et al.  A mid-level representation framework for semantic sports video analysis , 2003, ACM Multimedia.

[32]  Rudinei Goularte,et al.  Video scene segmentation by improved visual shot coherence , 2013, WebMedia.

[33]  Jenq-Neng Hwang,et al.  Fast and automatic video object segmentation and tracking for content-based applications , 2002, IEEE Trans. Circuits Syst. Video Technol..

[34]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[35]  Thomas Martin Deserno,et al.  Feature description with SIFT, SURF, BRIEF, BRISK, or FREAK? A general question answered for bone age assessment , 2016, Comput. Biol. Medicine.

[36]  Pablo Moreno-Ger,et al.  Game Learning Analytics: Learning Analytics for Serious Games , 2016 .

[37]  Yang Song,et al.  Taxonomic classification for web-based videos , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[39]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Brendan McCane,et al.  Better than SIFT? , 2015, Machine Vision and Applications.

[41]  Marcel Worring,et al.  On the surplus value of semantic video analysis beyond the key frame , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[42]  Stephanie de Smale.  Memory in the margins: The connecting and colliding of vernacular war memories , 2020, Media, War & Conflict.

[43]  Ina Blümel,et al.  Automatic Annotation of Scientific Video Material based on Visual Concept Detection , 2013, i-Know '13.

[44]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[45]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[46]  Rong Yan,et al.  Merging storyboard strategies and automatic retrieval for improving interactive video search , 2007, CIVR '07.

[47]  Chuan Wu,et al.  Events recognition by semantic inference for sports video , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[48]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[49]  Shih-Fu Chang,et al.  Structural and semantic analysis of video , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[50]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[51]  C. V. Jawahar,et al.  Automatic analysis of broadcast football videos using contextual priors , 2017, Signal Image Video Process..

[52]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Vasileios Mezaris,et al.  Fast shot segmentation combining global and local visual descriptors , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[54]  Mark Sanderson,et al.  Automatic video tagging using content redundancy , 2009, SIGIR.