The Multi-strand Graph for a PTZ Tracker

High-resolution images can be used to resolve matching ambiguities between trajectory fragments (tracklets), which is a key challenge in multiple-target tracking. A pan–tilt–zoom (PTZ) camera, which can pan, tilt and zoom, is a powerful and efficient tool that offers both close-up views and wide area coverage on demand. The wide area enables tracking of many targets, while the close-up view allows individuals to be identified from high-resolution images of their faces. A central component of a PTZ tracking system is a scheduling algorithm that determines which target to zoom in on, particularly when the high-resolution images are also used for tracklet matching. In this paper, we study this scheduling problem from a theoretical perspective. We propose a novel data structure, the Multi-strand Tracking Graph (MSG), which represents the set of tracklets computed by a tracker and the possible associations between them. The MSG allows efficient scheduling as well as resolving of matching ambiguities between tracklets. The main feature of the MSG is the auxiliary data saved in each vertex, which allows efficient computation while avoiding time-consuming graph traversal. Synthetic data simulations are used to evaluate our scheduling algorithm and to demonstrate its superiority over a naïve one.

[1]  Rui Caseiro,et al.  Globally optimal solution to multi-object tracking with merged measurements , 2011, 2011 International Conference on Computer Vision.

[2]  Pascal Fua,et al.  Tracking Interacting Objects Optimally Using Integer Programming , 2014, ECCV.

[3]  Ting Yu,et al.  Collaborative Real-Time Control of Active Cameras in Large Scale Surveillance Systems , 2008 .

[4]  C. Diehl,et al.  Scheduling an active camera to observe people , 2004, VSSN '04.

[5]  Michael D. Naish,et al.  Scheduling active camera resources for multiple moving targets , 2009, 2009 Canadian Conference on Electrical and Computer Engineering.

[6]  Mohan S. Kankanhalli,et al.  Decision-theoretic approach to maximizing observation of multiple targets in multi-camera surveillance , 2012, AAMAS.

[7]  Stefan Carlsson,et al.  Multi-Target Tracking - Linking Identities using Bayesian Network Inference , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[8]  Demetri Terzopoulos,et al.  Surveillance in Virtual Reality: System Design and Multi-Camera Control , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Andrew D. Bagdanov,et al.  Acquisition of high-resolution images through on-line saccade sequence planning , 2005 .

[10]  Margrit Betke,et al.  Efficient track linking methods for track graphs using network-flow and set-cover techniques , 2011, CVPR 2011.

[11]  Alberto Del Bimbo,et al.  Information theoretic sensor management for multi-target tracking with a single pan-tilt-zoom camera , 2014, IEEE Winter Conference on Applications of Computer Vision.

[12]  Amit K. Roy-Chowdhury,et al.  Distributed Constrained Optimization for Bayesian Opportunistic Visual Sensing , 2014, IEEE Transactions on Control Systems Technology.

[13]  Larry S. Davis,et al.  Constructing task visibility intervals for video surveillance , 2006, Multimedia Systems.

[14]  Gérard G. Medioni,et al.  Persistent people tracking and face capture using a PTZ camera , 2013, 2013 Seventh International Conference on Distributed Smart Cameras (ICDSC).

[15]  Demetri Terzopoulos,et al.  Planning ahead for PTZ camera assignment and handoff , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[16]  Rafael Medina Carnicer,et al.  Three hypothesis algorithm with occlusion reasoning for multiple people tracking , 2015, J. Electronic Imaging.

[17]  Taiki Sekii.  Robust, Real-Time 3D Tracking of Multiple Objects with Similar Appearances , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Hugo Proença,et al.  Dynamic camera scheduling for visual surveillance in crowded scenes using Markov random fields , 2015, 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[19]  Gérard G. Medioni,et al.  Inferring tracklets for multi-object tracking , 2011, CVPR 2011 WORKSHOPS.

[20]  Stefan Carlsson,et al.  Tracking and Labelling of Interacting Multiple Targets , 2006, ECCV.

[21]  Alberto Del Bimbo,et al.  Towards on-line saccade planning for high-resolution image sensing , 2006, Pattern Recognit. Lett..

[22]  Alberto Del Bimbo,et al.  Non-myopic information theoretic sensor management of a single pan-tilt-zoom camera for multiple object detection and tracking , 2015, Comput. Vis. Image Underst..

[23]  Gérard G. Medioni,et al.  Towards a practical PTZ face detection and tracking system , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[24]  Ramakant Nevatia,et al.  An online learned CRF model for multi-target tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Demetri Terzopoulos,et al.  Surveillance camera scheduling: a virtual vision approach , 2005, Multimedia Systems.

[26]  Peter H. Tu,et al.  Face recognition at a distance system for surveillance applications , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[27]  Larry S. Davis,et al.  Task Scheduling in Large Camera Networks , 2007, ACCV.

[28]  Pablo O. Arambel,et al.  Multiple-hypothesis tracking of multiple ground targets from aerial video with dynamic sensor control , 2004, SPIE Defense + Commercial Sensing.

[29]  Yael Moses,et al.  The multi-strand graph for a PTZ tracker , 2015, AVSS.

[30]  I-Jeng Wang,et al.  Surveillance Camera Coordination Through Distributed Scheduling , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[31]  Eric Sommerlade,et al.  Information-theoretic active scene exploration , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.