Fast and Accurate Object Detection Using Image Cropping/Resizing in Multi-View 4K Sports Videos

Recently, fast and accurate DNN object detectors such as YOLO and SSD have attracted considerable attention. However, it still takes far more time than real time processing when inputting a 4K video and becomes even more challenging when inputting multi-view 4K sports videos due to the massive amount of data and small objects. This paper presents a novel approach to significantly accelerate object detection. Observing the fact that object regions (including players and balls) are very sparse in sports videos, we crop the images to greatly reduce the processing areas using the temporal or view correlations. However, simple image cropping may worsen the accuracy if the object is too small to be detected. A further observation is that the detection can be improved by resizing the small objects properly. Our experimental results on two soccer matches of J1 League demonstrate that we can make the detection much faster with a high accuracy for small objects.

[1]  Paramvir Bahl,et al.  Real-Time Video Analytics: The Killer App for Edge Computing , 2017, Computer.

[2]  Tae-Hyun Oh,et al.  Part-Based Player Identification Using Deep Convolutional Representation and Multi-scale Pooling , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Dong Xu,et al.  Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey , 2018, IEEE Signal Processing Magazine.

[4]  Huang-Chia Shih,et al.  A Survey of Content-Aware Video Analysis for Sports , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Thomas S. Huang,et al.  Survey of Face Detection on Low-Quality Images , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[6]  Sridha Sridharan,et al.  Predicting Ball Ownership in Basketball from a Monocular View Using Only Player Trajectories , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[7]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Bruno Volckaert,et al.  Embedded Real-Time Object Detection for a UAV Warning System , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[9]  Mahadev Satyanarayanan,et al.  Enabling Live Video Analytics with a Scalable and Privacy-Aware Framework , 2018, ACM Trans. Multim. Comput. Commun. Appl..

[10]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[11]  Hideo Saito,et al.  Ball 3D Trajectory Reconstruction without Preliminary Temporal and Geometrical Camera Calibration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[13]  Vito Renó,et al.  Convolutional Neural Networks Based Ball Detection in Tennis Games , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Bir Bhanu,et al.  Soccer: Who Has the Ball? Generating Visual Analytics and Player Statistics , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Sergio Guadarrama,et al.  Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Peiyun Hu,et al.  Finding Tiny Faces , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[19]  Bernard Ghanem,et al.  Finding Tiny Faces in the Wild with Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Bing-Yu Chen,et al.  High‐resolution 360 Video Foveated Stitching for Real‐time VR , 2017, Comput. Graph. Forum.

[21]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Ira Kemelmacher-Shlizerman,et al.  Soccer on Your Tabletop , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Miran Pobar,et al.  Object detection in sports videos , 2018, 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[24]  Ishfaq Ahmad.  Multi-View Video: Get Ready for Next-Generation Television , 2007, IEEE Distributed Systems Online.