Modeling Correspondences for Multi-Camera Tracking Using Nonlinear Manifold Learning and Target Dynamics

Multi-camera tracking systems often must maintain consistent identity labels of the targets across views to recover 3D trajectories and fully take advantage of the additional information available from the multiple sensors. Previous approaches to the "correspondence across views" problem include matching features, using camera calibration information, and computing homographies between views under the assumption that the world is planar. However, it can be difficult to match features across significantly different views. Furthermore, calibration information is not always available and planar world hypothesis can be too restrictive. In this paper, a new approach is presented for matching correspondences based on the use of nonlinear manifold learning and system dynamics identification. The proposed approach does not require similar views, calibration nor geometric assumptions of the 3D environment, and is robust to noise and occlusion. Experimental results demonstrate the use of this approach to generate and predict views in cases where identity labels become ambiguous.

[1]  L. Davis,et al.  M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene , 2003, International Journal of Computer Vision.

[2]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[3]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[4]  Jake K. Aggarwal,et al.  Tracking Human Motion in Structured Environments Using a Distributed-Camera System , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Ahmed M. Elgammal.  Nonlinear Generative Models for Dynamic Shape and Dynamic Appearance , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[6]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[7]  Mubarak Shah,et al.  Consistent Labeling of Tracked Objects in Multiple Cameras with Overlapping Fields of View , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Mario Sznaier,et al.  Open-loop worst-case identification of nonSchur plants , 2003, Autom..

[9]  A. Elgammal,et al.  Separating style and content on a nonlinear manifold , 2004, CVPR 2004.

[10]  M. Brand.  From Subspaces to Submanifolds , 2004 .

[11]  Matthew E. Brand.  Subspace mappings for image sequences , 2002 .

[12]  A. M. Tekalp,et al.  Multiple camera tracking of interacting and occluded human motion , 2001, Proc. IEEE.

[13]  Michael J. Black,et al.  Robust Principal Component Analysis for Computer Vision , 2001, ICCV.

[14]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[15]  Luc Van Gool,et al.  Color-Based Object Tracking in Multi-camera Environments , 2003, DAGM-Symposium.

[16]  Daniel D. Lee,et al.  Learning High Dimensional Correspondences from Low Dimensional Manifolds , 2003 .

[17]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[18]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[19]  Lily Lee,et al.  Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Stefano Soatto,et al.  Dynamic Textures , 2003, International Journal of Computer Vision.

[21]  Matthew Brand,et al.  Continuous nonlinear dimensionality reduction by kernel Eigenmaps , 2003, IJCAI.

[22]  Daniel D. Lee,et al.  Semisupervised alignment of manifolds , 2005, AISTATS.

[23]  H.F. Durrant-Whyte,et al.  A new approach for filtering nonlinear systems , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[24]  Michael J. Black,et al.  Robust parameterized component analysis: theory and applications to 2D facial appearance models , 2003, Comput. Vis. Image Underst..

[25]  Kilian Q. Weinberger,et al.  Learning a kernel matrix for nonlinear dimensionality reduction , 2004, ICML.

[26]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.

[28]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, CVPR.

[29]  Matthew Brand.  From Subspace to Submanifold Methods , 2004, BMVC.

[30]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[31]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[32]  Mario Sznaier,et al.  A Caratheodory-Fejer approach to robust multiframe tracking , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[33]  Michael Isard,et al.  Learning and Classification of Complex Dynamics , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Takeo Kanade,et al.  An active camera system for acquiring multi-view video , 2002, Proceedings. International Conference on Image Processing.

[35]  Michael J. Black,et al.  Dynamic coupled component analysis , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[36]  Yaron Caspi,et al.  A step towards sequence-to-sequence alignment , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[37]  Matthew Brand,et al.  Charting a Manifold , 2002, NIPS.

[38]  Nikos A. Vlassis,et al.  Non-linear CCA and PCA by Alignment of Local Models , 2003, NIPS.

[39]  LeeLily,et al.  Monitoring Activities from Multiple Video Streams , 2000 .

[40]  Shaogang Gong,et al.  Tracking multiple people with a multi-camera system , 2001, Proceedings 2001 IEEE Workshop on Multi-Object Tracking.

[41]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[42]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[43]  A. Elgammal,et al.  Inferring 3D body pose from silhouettes using activity manifold learning , 2004, CVPR 2004.

[44]  Dorin Comaniciu,et al.  Adaptive Resolution System for Distributed Surveillance , 2002, Real Time Imaging.

[45]  Manolis I. A. Lourakis,et al.  Real-Time Tracking of Multiple Skin-Colored Objects with a Possibly Moving Camera , 2004, ECCV.