Automated repair of fragmented tracks with 1D CNNs

Abstract Multiple object tracking is an important but challenging computer vision problem. The complex motion of objects makes tracking difficult during long periods of object occlusion, and as a result occlusions frequently cause fragmented tracks with gaps. Previous works use linear interpolation to fill in such gaps, a technique which is only able to model simple motion. As a result, tracked bounding box locations can be quite poor in these situations. In this paper, we propose a 1D CNN based solution to filling gaps which models complex motion in a data-driven way. Our proposed solution uses only bounding box coordinates as input, and as such does not incur the computational cost of processing image features directly. We show that our model significantly outperforms linear interpolation on dynamic sports datasets in terms of mean intersection over union between predicted and ground truth bounding boxes.

[1]  Afshin Dehghan,et al.  GMCP-Tracker: Global Multi-object Tracking Using Generalized Minimum Clique Graphs , 2012, ECCV.

[2]  Nobuyuki Yagi,et al.  Probabilistic Integration of Tracking and Recognition of Soccer Players , 2009, MMM.

[3]  Laura Leal-Taixé,et al.  Tracking Without Bells and Whistles , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[4]  Konrad Schindler,et al.  Continuous Energy Minimization for Multitarget Tracking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Kuk-Jin Yoon,et al.  Robust Online Multi-object Tracking Based on Tracklet Confidence and Online Discriminative Appearance Learning , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Silvio Savarese,et al.  SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Andrew Zisserman,et al.  Detect to Track and Track to Detect , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Jenq-Neng Hwang,et al.  Exploit the Connectivity: Multi-Object Tracking with TrackletNet , 2018, ACM Multimedia.

[9]  Tae-Hyun Oh,et al.  Part-Based Player Identification Using Deep Convolutional Representation and Multi-scale Pooling , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Luis E. Ortiz,et al.  Who are you with and where are you going? , 2011, CVPR 2011.

[11]  Oliver Grau,et al.  Tracking football player movement from a single moving camera using particle filters , 2006 .

[12]  Konrad Schindler,et al.  Learning by Tracking: Siamese CNN for Robust Target Association , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[13]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Irfan A. Essa,et al.  Player localization using multiple static cameras for sports visualization , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Michael Beetz,et al.  An Adaptive Vision System for Tracking Soccer Players from Variable Camera Settings , 2007, ICVS 2007.

[16]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[17]  Pinar Duygulu Sahin,et al.  Sentioscope: A Soccer Player Tracking System Using Model Field Particles , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Stefan Roth,et al.  MOT16: A Benchmark for Multi-Object Tracking , 2016, ArXiv.

[19]  Silvio Savarese,et al.  Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Wei Xu,et al.  Tracklet Association Tracker: An End-to-End Learning-based Association Approach for Multi-Object Tracking , 2018, ArXiv.

[21]  Behzad Dariush,et al.  Looking to Relations for Future Trajectory Forecast , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[22]  Bernt Schiele,et al.  Multiple People Tracking by Lifted Multicut and Person Re-identification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Rama Chellappa,et al.  A Multiple-Hypothesis Approach for Multiobject Visual Tracking , 2007, IEEE Transactions on Image Processing.

[24]  Hamid Abrishami Moghaddam,et al.  A survey on player tracking in soccer videos , 2017, Comput. Vis. Image Underst..

[25]  Silvio Savarese,et al.  Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[27]  Anderson Rocha,et al.  A multiple camera methodology for automatic localization and tracking of futsal players , 2014, Pattern Recognit. Lett..

[28]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[29]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[30]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Donald Reid.  An algorithm for tracking multiple targets , 1978 .

[32]  Hitoshi Sato,et al.  Multiple sports player tracking system based on graph optimization using low-cost cameras , 2018, 2018 IEEE International Conference on Consumer Electronics (ICCE).

[33]  Hiroshi Sankoh,et al.  Automatic Soccer Player Tracking in Single Camera with Robust Occlusion Handling Using Attribute Matching , 2015, IEICE Trans. Inf. Syst..

[34]  Yanxi Liu,et al.  Tracking Sports Players with Context-Conditioned Motion Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  James M. Rehg,et al.  Multi-object Tracking with Neural Gating Using Bilinear LSTM , 2018, ECCV.