Fine-Grained Visual Dribbling Style Analysis for Soccer Videos With Augmented Dribble Energy Image

Recent advances in interpretations of soccer are predominantly made through analyzing high-level contents of soccer videos. This work targets on these highlight actions and movements in soccer games and it focuses on dribbling skills performed by the top players. Our work leverages understanding of complex dribbling video clips by representing a video sequence with a single Dribble Energy Image(DEI) that is informative for dribbling styles recognition. To overcome the shortage of labelled data, this paper introduces a dataset of soccer video clips from Youtube, employs Mask-RCNN to segment out dribbling players and OpenPose to obtain joints information of dribbling players. Besides, to solve issues caused by camera motions in highlight soccer videos, our work proposes to register a video sequence to generate a single image representation DEI and dribbling styles classification. Our approach can achieve an accuracy of 87.65% on dribbling styles classification and it is observed that data augmentation using joints-reasoned GAN can improve the classification performance.

[1]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[4]  Andrea Vedaldi,et al.  Dynamic Image Networks for Action Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Jing Xue,et al.  Automatic Soccer Video Event Detection Based on a Deep Neural Network Combined CNN and RNN , 2016, 2016 IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI).

[6]  Tatsuya Harada,et al.  Football Action Recognition Using Hierarchical LSTM , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[9]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[10]  Christian Wolf,et al.  Action Classification in Soccer Videos with Long Short-Term Memory Recurrent Neural Networks , 2010, ICANN.

[11]  Bir Bhanu,et al.  Soccer: Who Has the Ball? Generating Visual Analytics and Player Statistics , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[13]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[14]  Marc Van Droogenbroeck,et al.  A Bottom-Up Approach Based on Semantics for the Interpretation of the Main Camera Stream in Soccer Games , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[18]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[19]  Jitendra Malik,et al.  Recognizing action at a distance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[21]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.