Automatic Basketball Detection in Sport Video Based on R-FCN and Soft-NMS

In basketball videos, the ball is always so small in the camera that its appearance feature is hard to be extracted. In this paper, we introduce a deep-learning technology to detect the basketball. Specifically, we train our basketball detection model based on the Region-based Fully Convolutional Networks (R-FCN) which uses the fully convolutional Residual Network (ResNet) as the backbone network. What's more, we apply some new techniques including Online Hard Example Mining (OHEM), Soft-NMS and multi-scale training strategy to achieve higher detection accuracy. In detail, the OHEM method can reduce the cost of fine-tuning during training by calculating the loss of the RoIs. Soft-NMS can reduce the false positive rate by decreasing the object detection score between the overlap object. And the multi-scale training can improve the detection performance by receiving the good feature from the object with different scale. Finally, we achieve a mean average precision (mAP) value of 89.7% on a public basketball dataset. It proves that applying the deep-learning approach to basketball detection is effective.

[1]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[2]  Pascal Fua,et al.  What Players do with the Ball: A Physically Constrained Interaction Modeling , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Hua-Tsung Chen,et al.  Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video , 2009, J. Vis. Commun. Image Represent..

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[6]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Abhinav Gupta,et al.  Training Region-Based Object Detectors with Online Hard Example Mining , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yongsheng Zhao,et al.  A learning framework towards real-time detection and localization of a ball for robotic table tennis system , 2017, 2017 IEEE International Conference on Real-time Computing and Robotics (RCAR).

[10]  Cataldo Guaragnella,et al.  A new algorithm for ball recognition using circle Hough transform and neural classifier , 2004, Pattern Recognit..

[11]  Larry S. Davis,et al.  Soft-NMS — Improving Object Detection with One Line of Code , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[13]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[14]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[15]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[16]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[17]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  M. Archana,et al.  Object Detection and Tracking Based on Trajectory in Broadcast Tennis Video , 2015 .

[19]  Brejesh Lall,et al.  Automated ball tracking in tennis videos , 2015, 2015 Third International Conference on Image Information Processing (ICIIP).

[20]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Sukadev Meher,et al.  A real-time trajectory-based ball detection-and-tracking framework for basketball video , 2013 .

[22]  Josef Halbinger,et al.  Video-Based Soccer Ball Detection in Difficult Situations , 2013, icSPORTS 2013.

[23]  Pascal Fua,et al.  Take your eyes off the ball: Improving ball-tracking by focusing on team play , 2014, Comput. Vis. Image Underst..