Spatial-temporal consistent labeling of tracked pedestrians across non-overlapping camera views

Tracking people across multiple cameras with non-overlapping views is a challenging task, since their observations are separated in time and space and their appearances may vary significantly. This paper proposes a Bayesian model to solve the consistent labeling problem across multiple non-overlapping camera views. Significantly different from related approaches, our model assumes neither people are well segmented nor their trajectories across camera views are estimated. We formulate a spatial-temporal probabilistic model in the hypothesis space that consists the potentially matched objects between the exit field of view (FOV) of one camera and the entry FOV of another camera. A competitive major color spectrum histogram representation (CMCSHR) for appearance matching between two objects is also proposed. The proposed spatial-temporal and appearance models are unified by a maximum-a-posteriori (MAP) Bayesian model. Based on this Bayesian model, when a detected new object corresponds to a group hypothesis (more than one object), we further develop an online method for online correspondence update using optimal graph matching (OGM) algorithm. Experimental results on three different real scenarios validate the proposed Bayesian model approach and the CMCSHR method. The results also show that the proposed approach is able to address the occlusion problem/group problem, i.e. finding the corresponding individuals in another camera view for a group of people who walk together into the entry FOV of a camera.

[1]  Xiaogang Wang,et al.  Shape and Appearance Context Modeling , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[2]  Tim J. Ellis,et al.  Bridging the gaps between cameras , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[3]  Simone Calderara,et al.  HECOL: Homography and epipolar-based consistent labeling for outdoor park surveillance , 2008, Comput. Vis. Image Underst..

[4]  Michael J. Swain,et al.  Indexing via color histograms , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[5]  Ramin Zabih,et al.  Bayesian multi-camera surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[6]  Zhi-Qiang Liu,et al.  Self-splitting competitive learning: a new on-line clustering paradigm , 2002, IEEE Trans. Neural Networks.

[7]  David G. Stork,et al.  Pattern Classification , 1973 .

[8]  Massimo Piccardi,et al.  Tracking people across disjoint camera views by an illumination-tolerant appearance representation , 2007, Machine Vision and Applications.

[9]  Hai Tao,et al.  Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features , 2008, ECCV.

[10]  Christopher O. Jaynes,et al.  Object matching in disjoint cameras using a color transfer approach , 2007, Machine Vision and Applications.

[11]  W. Eric L. Grimson,et al.  Correspondence-Free Activity Analysis and Scene Modeling in Multiple Camera Views , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Patrick Pérez,et al.  Sequential Monte Carlo methods for multiple target tracking and data fusion , 2002, IEEE Trans. Signal Process..

[13]  Mubarak Shah,et al.  Modeling inter-camera space-time and appearance relationships for tracking across non-overlapping views , 2008, Comput. Vis. Image Underst..

[14]  Trevor Darrell,et al.  Simultaneous calibration and tracking with a network of non-overlapping sensors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[15]  Francisco José Madrid-Cuevas,et al.  Particle filtering with multiple and heterogeneous cameras , 2010, Pattern Recognit..

[16]  Yu-ming Cheung.  Rival penalization controlled competitive learning for data clustering with unknown cluster number , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[17]  Massimo Piccardi,et al.  Disjoint track matching based on a major color spectrum histogram representation , 2007 .

[18]  Andrew Gilbert,et al.  Tracking Objects Across Cameras by Incrementally Learning Inter-camera Colour Calibration and Patterns of Activity , 2006, ECCV.

[19]  Joon Hee Han,et al.  Object handoff between uncalibrated views without planar ground assumption , 2008, Pattern Recognit. Lett..

[20]  Yi-Ping Hung,et al.  An adaptive learning method for target tracking across multiple cameras , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Aristidis Likas,et al.  The global kernel k-means clustering algorithm , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[22]  Richard I. Hartley,et al.  Person Reidentification Using Spatiotemporal Appearance , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  Shaogang Gong,et al.  Associating Groups of People , 2009, BMVC.

[24]  Larry S. Davis,et al.  M2Tracker: A Multi-view Approach to Segmenting and Tracking People in a Cluttered Scene Using Region-Based Stereo , 2002, ECCV.

[25]  Mubarak Shah,et al.  Consistent Labeling of Tracked Objects in Multiple Cameras with Overlapping Fields of View , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[28]  Xiaojun Wan,et al.  A New Retrieval Model Based on TextTiling for Document Similarity Search , 2005, Journal of Computer Science and Technology.

[29]  Simone Calderara,et al.  Bayesian-Competitive Consistent Labeling for People Surveillance , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Shaogang Gong,et al.  Multi-camera Matching using Bi-Directional Cumulative Brightness Transfer Functions , 2008, BMVC.

[31]  Luc Van Gool,et al.  Color-Based Object Tracking in Multi-camera Environments , 2003, DAGM-Symposium.

[32]  Nikos A. Vlassis,et al.  The global k-means clustering algorithm , 2003, Pattern Recognit..

[33]  Tieniu Tan,et al.  Principal axis-based correspondence between multiple cameras for people tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Mubarak Shah,et al.  Appearance modeling for tracking in multiple non-overlapping cameras , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[35]  E. Forgy,et al.  Cluster analysis of multivariate data : efficiency versus interpretability of classifications , 1965 .

[36]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.