Player Tracking and Identification in Ice Hockey

Tracking and identifying players is a fundamental step in computer vision-based ice hockey analytics. The data generated by tracking is used in many other downstream tasks, such as game event detection and game strategy analysis. Player tracking and identification is a challenging problem since the motion of players in hockey is fast-paced and non-linear when compared to pedestrians. There is also significant camera panning and zooming in hockey broadcast video. Identifying players in ice hockey is challenging since the players of the same team look almost identical, with the jersey number the only discriminating factor between players. In this paper, an automated system to track and identify players in broadcast NHL hockey videos is introduced. The system is composed of three components (1) Player tracking, (2) Team identification and (3) Player identification. Due to the absence of publicly available datasets, the datasets used to train the three components are annotated manually. Player tracking is performed with the help of a state of the art tracking algorithm obtaining a Multi-Object Tracking Accuracy (MOTA) score of 94.5%. For team identification, the away-team jerseys are grouped into a single class and hometeam jerseys are grouped in classes according to their jersey color. A convolutional neural network is then trained on the team identification dataset. The team identification network gets an accuracy of 97% on the test set. A novel player identification model is introduced that utilizes a temporal one-dimensional convolutional network to identify players from player bounding box sequences. The player identification model further takes advantage of the available NHL game roster data to obtain a player identification accuracy of 83%.

[1]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[3]  Tiziana D'Orazio,et al.  An Investigation Into the Feasibility of Real-Time Soccer Offside Detection From a Multiple Camera System , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Siavash Gorji,et al.  Group Activity Detection from Trajectory and Video Data in Soccer , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Martin D. Levine,et al.  Player Identification in Hockey Broadcast Videos , 2020, Expert Syst. Appl..

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  Karsten Müller,et al.  Soccer Jersey Number Recognition Using Convolutional Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[9]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Changhu Wang,et al.  Jersey Number Recognition with Semi-Supervised Spatial Transformer Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  Tianxiao Guo,et al.  Detection of Ice Hockey Players and Teams via a Two-Phase Cascaded CNN Model , 2020, IEEE Access.

[12]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[13]  Francesco Solera,et al.  Performance Measures and a Data Set for Multi-target, Multi-camera Tracking , 2016, ECCV Workshops.

[14]  James J. Little,et al.  Learning to Track and Identify Players from Broadcast Sports Videos , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Laura Leal-Taix'e,et al.  Learning a Neural Solver for Multiple Object Tracking , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Gloria Haro,et al.  Single-Camera Basketball Tracker through Pose and Semantic Feature Fusion , 2019, ArXiv.

[17]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Karsten Müller,et al.  Soccer player recognition using spatial constellation features and jersey number recognition , 2017, Comput. Vis. Image Underst..

[19]  Yukun Yang,et al.  Multi-camera multi-player tracking with deep player identification in sports video , 2020, Pattern Recognit..

[20]  Jingchen Liu,et al.  Detecting and Tracking Sports Players with Random Forests and Context-Conditioned Motion Models , 2014 .

[21]  Pascal Fua,et al.  Tracking multiple people under global appearance constraints , 2011, 2011 International Conference on Computer Vision.

[22]  Bir Bhanu,et al.  Pose-Guided R-CNN for Jersey Number Recognition in Sports , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[23]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Tiziana D'Orazio,et al.  Visual Players Detection and Tracking in Soccer Matches , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.

[25]  Sridha Sridharan,et al.  Representing Team Behaviours from Noisy Data Using Player Role , 2014, CVPR 2014.

[26]  Gloria Haro,et al.  Self-Supervised Small Soccer Player Detection and Tracking , 2020, ArXiv.

[27]  LiuJia,et al.  Automatic player labeling, tracking and field registration and trajectory mapping in broadcast soccer video , 2011 .

[28]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[29]  David A. Clausi,et al.  Puck localization and multi-task event recognition in broadcast hockey videos , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  James J. Little,et al.  A Boosted Particle Filter: Multitarget Detection and Tracking , 2004, ECCV.

[31]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[32]  Zdravko Ivankovic,et al.  Automatic player position detection in basketball games , 2013, Multimedia Tools and Applications.

[33]  Bir Bhanu,et al.  An Automated System for Generating Tactical Performance Statistics for Individual Soccer Players From Videos , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Hrvoje Dujmić,et al.  Player Number Localization and Recognition in Soccer Video using HSV Color Space and Internal Contours , 2008 .

[35]  Tae-Hyun Oh,et al.  Part-Based Player Identification Using Deep Convolutional Representation and Multi-scale Pooling , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[36]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[37]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[38]  Bernt Schiele,et al.  Multiple People Tracking by Lifted Multicut and Person Re-identification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  David A. Clausi,et al.  Multi-task Learning for Jersey Number Recognition in Ice Hockey , 2021, MMSports@MM.

[40]  James H. Elder,et al.  Contrastive Learning for Sports Video: Unsupervised Player Classification , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[41]  David A. Clausi,et al.  Event detection in coarsely annotated sports videos via parallel multi receptive field 1D convolutions , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[42]  Fabio Tozeto Ramos,et al.  Simple online and realtime tracking , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[43]  Christophe De Vleeschouwer,et al.  Associative Embedding for Team Discrimination , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[44]  Jia Liu,et al.  Automatic player labeling, tracking and field registration and trajectory mapping in broadcast soccer video , 2011, TIST.

[45]  Wenjun Zeng,et al.  FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking. , 2020 .

[46]  Laura Leal-Taixé,et al.  Tracking Without Bells and Whistles , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[47]  Patrick Pérez,et al.  Maintaining multimodality through mixture tracking , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[48]  James J. Little,et al.  Robust Visual Tracking for Multiple Targets , 2006, ECCV.