Combining multiple scoring systems for target tracking using rank-score characteristics

Video target tracking is the process of estimating the current state, and predicting the future state of a target from a sequence of video sensor measurements. Multitarget video tracking is complicated by the fact that targets can occlude one another, affecting video feature measurements in a highly non-linear and difficult to model fashion. In this paper, we apply a multisensory fusion approach to the problem of multitarget video tracking with occlusion. The approach is based on a data-driven method (CFA) to selecting the features and fusion operations that improve a performance criterion. Each sensory cue is treated as a scoring system. Scoring behavior is characterized by a rank-score function. A diversity measure, based on the variation in rank-score functions, is used to dynamically select the scoring systems and fusion operations that produce the best tracking performance. The relationship between the diversity measure and the tracking accuracy of two fusion operations, a linear score combination and an average rank combination, is evaluated on a set of 12 video sequences. These results demonstrate that using the rank-score characteristic as a diversity measure is an effective method to dynamically select scoring systems and fusion operations that improve the performance of multitarget video tracking with occlusions.

[1]  Pramod K. Varshney,et al.  Sensor Fusion for Video Surveillance , 2004 .

[2]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[4]  D. Frank Hsu,et al.  A study of data fusion in Cayley graphs G(s/sub n/,p/sub n/) , 2004, 7th International Symposium on Parallel Architectures, Algorithms and Networks, 2004. Proceedings..

[5]  Kagan Tumer,et al.  Linear and Order Statistics Combiners for Pattern Classification , 1999, ArXiv.

[6]  Ofer Melnik,et al.  Mixed group ranks: preference and confidence in classifier combination , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Damian M. Lyons,et al.  A dynamic pruning and feature selection strategy for real-time tracking , 2005, 19th International Conference on Advanced Information Networking and Applications (AINA'05) Volume 1 (AINA papers).

[8]  Yaakov Bar-Shalom,et al.  Multitarget-Multisensor Tracking , 1995 .

[9]  Damian M. Lyons,et al.  Experimental results from using a rank and fuse approach for multi-target tracking in CCTV surveillance , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[10]  Paul R. Schrater,et al.  Bayesian data fusion and credit assignment in vision and fMRI data analysis , 2003, IS&T/SPIE Electronic Imaging.

[11]  Josef Kittler,et al.  Sum Versus Vote Fusion in Multiple Classifier Systems , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Damian M. Lyons,et al.  Selecting and Evaluating Combinatorial Fusion Criteria to Improve Multitarget Tracking , 2006, 2006 9th International Conference on Information Fusion.

[14]  Nageswara S. V. Rao.  A Generic Sensor Fusion Problem: Classification and Function Estimation , 2004, Multiple Classifier Systems.

[15]  Damian M. Lyons,et al.  RAF: a dynamic and efficient approach to fusion for multitarget tracking in CCTV surveillance , 2003, Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI2003..

[16]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Jochen Triesch,et al.  Democratic Integration: Self-Organized Integration of Adaptive Cues , 2001, Neural Computation.

[18]  D. Frank Hsu,et al.  Consensus Scoring Criteria for Improving Enrichment in Virtual Screening , 2005, J. Chem. Inf. Model..

[19]  Luke Fletcher,et al.  An adaptive fusion architecture for target tracking , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[20]  Gregory D. Hager,et al.  Joint probabilistic techniques for tracking multi-part objects , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[21]  William H. Press,et al.  Numerical recipes in C , 2002 .

[22]  Moni Naor,et al.  Rank aggregation methods for the Web , 2001, WWW '01.

[23]  Nageswara S. V. Rao,et al.  Finite sample performance guarantees of fusers for function estimators , 2000, Inf. Fusion.

[24]  Larry S. Davis,et al.  W/sup 4/: Who? When? Where? What? A real time system for detecting and tracking people , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[25]  Pramod K. Varshney,et al.  Special issue on data fusion , 1997 .

[26]  Chuan Yi Tang,et al.  Feature Selection and Combination Criteria for Improving Accuracy in Protein Structure Prediction , 2007, IEEE Transactions on NanoBioscience.

[27]  P. Reisman,et al.  Crowd detection in video sequences , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[28]  D. Frank Hsu,et al.  Comparing Rank and Score Combination Methods for Data Fusion in Information Retrieval , 2005, Information Retrieval.

[29]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[30]  Robert A. Jacobs,et al.  Modeling the Combination of Motion, Stereo, and Vergence Angle Cues to Visual Depth , 1999, Neural Computation.

[31]  Belur V. Dasarathy,et al.  Elucidative fusion systems - an exposition , 2000, Inf. Fusion.

[32]  P. Verlinde,et al.  Multi-level Data Fusion for the Detection of Targets Using Multi-spectral Image Sequences , 1998 .

[33]  D. Frank Hsu,et al.  Combinatorial Fusion Analysis: Methods and Practices of Combining Multiple Scoring Systems , 2006 .

[34]  Josef Kittler,et al.  Fusion of multiple classifiers , 2002, Inf. Fusion.

[35]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[36]  Ingemar J. Cox,et al.  An efficient implementation and evaluation of Reid's multiple hypothesis tracking algorithm for visual tracking , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[37]  K. Wilson,et al.  Person Tracking Using Audio-Video Sensor Fusion , 2001 .

[38]  Jake K. Aggarwal,et al.  Physics-based integration of multiple sensing modalities for scene interpretation , 1997 .

[39]  Ludmila I. Kuncheva.  Diversity in multiple classifier systems , 2005, Inf. Fusion.

[40]  Marc Acheroy,et al.  Multilevel data fusion for the detection of targets using multispectral image sequences , 1998 .

[41]  T. Leen,et al.  Probabilistic model-based multisensor image fusion , 1999 .

[42]  Paul B. Kantor,et al.  Predicting the effectiveness of Naïve data fusion on the basis of system characteristics , 2000 .

[43]  D. Frank Hsu,et al.  A Study of Data Fusion in Cayley Graphs G(S{n}, P{n}). , 2004 .

[44]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.