Contextual Tracking Approaches in Information Fusion

Many information fusion solutions work well in the intended scenarios; but the applications, supporting data, and capabilities change over varying contexts. One example is weather data for electro-optical target trackers of which standards have evolved over decades. The operating conditions of technology changes, sensor/target variations, and the contextual environment can inhibit performance if not included in the initial systems design. In this chapter, we seek to define and categorize different types of contextual information. We describe five contextual information categories that support target tracking: (1) domain knowledge from a user to aid the information fusion process through selection, cueing, and analysis, (2) environment-to-hardware processing for sensor management, (3) known distribution of entities for situation/threat assessment, (4) historical traffic behavior for situation awareness patterns of life (POL), and (5) road information for target tracking and identification. Appropriate characterization and representation of contextual information is needed for future high-level information fusion designs to take advantage of the large data content available for a priori knowledge target tracking algorithm construction, implementation, and application.

[1]  Erik Blasch,et al.  Data association through fusion of target track and identification sets , 2000, Proceedings of the Third International Conference on Information Fusion.

[2]  Wolfgang Koch,et al.  Road-map assisted convoy track maintenance using random matrices , 2008, 2008 11th International Conference on Information Fusion.

[3]  Stephen J. McKenna,et al.  Learning spatial context from tracking using penalised likelihoods , 2004, ICPR 2004.

[4]  Shengping Zhang,et al.  Robust visual tracking via context objects computing , 2011, 2011 18th IEEE International Conference on Image Processing.

[5]  Alan N. Steinberg,et al.  Situation and context in data fusion and natural language understanding , 2008, 2008 11th International Conference on Information Fusion.

[6]  E. Blasch,et al.  Sensor Management Fusion Using Operating Conditions , 2008, 2008 IEEE National Aerospace and Electronics Conference.

[7]  Li Bai,et al.  Multiple Kernel Learning for vehicle detection in wide area motion imagery , 2012, 2012 15th International Conference on Information Fusion.

[8]  C. Yang,et al.  Pose angular-aiding for maneuvering target tracking , 2005, 2005 7th International Conference on Information Fusion.

[9]  Wen-Huang Cheng,et al.  Generative and Discriminative Modeling toward Semantic Context Detection in Audio Tracks , 2005, 11th International Multimedia Modelling Conference.

[10]  Luis A. Hernández Gómez,et al.  Initialization, Training, and Context-Dependency in HMM-Based Formant Tracking , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[11]  Gang Hua,et al.  Context-Aware Visual Tracking , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Mohan M. Trivedi,et al.  Multiperspective Thermal IR and Video Arrays for 3D Body Tracking and Driver Activity Analysis , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[13]  Erik Blasch,et al.  Simultaneous feature-based identification and track fusion , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[14]  Erik Blasch,et al.  Pattern Activity Clustering and Evaluation (PACE) , 2012, Defense + Commercial Sensing.

[15]  Olga Mendoza-Schrock,et al.  Video image registration evaluation for a layered sensing environment , 2009, Proceedings of the IEEE 2009 National Aerospace & Electronics Conference (NAECON).

[16]  Li Bai,et al.  Efficient Minimum Error Bounded Particle Resampling L1 Tracker With Occlusion Detection , 2013, IEEE Transactions on Image Processing.

[17]  Shuo Zhang,et al.  Track Segment Association for GMTI Tracks of Evasive Move-Stop-Move Maneuvering Targets , 2011, IEEE Transactions on Aerospace and Electronic Systems.

[18]  Qiang Ji,et al.  Spatio-Temporal Context for Robust Multitarget Tracking , 2007 .

[19]  Jesús García,et al.  Research Opportunities in Contextualized Fusion Systems. The Harbor Surveillance Case , 2011, IWANN.

[20]  Yijun Yu,et al.  Iterative context-aware feature location: (NIER track) , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[21]  Guna Seetharaman,et al.  Efficient feature extraction and likelihood fusion for vehicle tracking in low frame rate airborne video , 2010, 2010 13th International Conference on Information Fusion.

[22]  Matthew E. P. Davies,et al.  Context-Dependent Beat Tracking of Musical Audio , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[23]  Erik Blasch,et al.  Sensor, User, Mission (SUM) Resource Management and Their Interaction with Level 2/3 Fusion , 2006, 2006 9th International Conference on Information Fusion.

[24]  Chun Yang,et al.  Mutual-aided target tracking and identification , 2003, SPIE Defense + Commercial Sensing.

[25]  Vikram Krishnamurthy,et al.  Detection of Anomalous Trajectory Patterns in Target Tracking via Stochastic Context-Free Grammars and Reciprocal Process Models , 2013, IEEE Journal of Selected Topics in Signal Processing.

[26]  Amiya Nayak,et al.  Adaptive and Intelligent Route Learning for Mobile Assets Using Geo-tracking and Context Profiles , 2009, 2009 International Conference on Computational Science and Engineering.

[27]  J. G. Herrero,et al.  Use of map information for tracking targets on airport surface , 2003 .

[28]  Branko Ristic,et al.  A variable structure multiple model particle filter for GMTI tracking , 2002, Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002. (IEEE Cat.No.02EX5997).

[29]  Erik Blasch,et al.  Issues and Challenges in Situation Assessment (Level 2 Fusion) , 2006, J. Adv. Inf. Fusion.

[30]  Gregory D. Abowd,et al.  Towards a Better Understanding of Context and Context-Awareness , 1999, HUC.

[31]  Erik Blasch,et al.  Level 5 (User Refinement) issues supporting Information Fusion Management , 2006, 2006 9th International Conference on Information Fusion.

[32]  Kannappan Palaniappan,et al.  Contemporary concerns in Geographical/Geospatial Information Systems (GIS) processing , 2011, Proceedings of the 2011 IEEE National Aerospace and Electronics Conference (NAECON).

[33]  Zhonghai Wang,et al.  Pattern of life from WAMI objects tracking based on visual context-aware tracking and infusion network models , 2013, Defense, Security, and Sensing.

[34]  Gian Luca Foresti,et al.  Markov Logic Networks for context integration and situation assessment in maritime domain , 2012, 2012 15th International Conference on Information Fusion.

[35]  Miguel A. Patricio,et al.  On the representation and exploitation of context knowledge in a harbor surveillance scenario , 2011, 14th International Conference on Information Fusion.

[36]  Erik Blasch,et al.  Situation, impact, and user refinement , 2003, SPIE Defense + Commercial Sensing.

[37]  Qiang Ji,et al.  Robust multi-target tracking using spatio-temporal context , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[38]  Jesús García,et al.  Context-based Information Fusion: A survey and discussion , 2015, Inf. Fusion.

[39]  Philippe Salembier,et al.  Segmentation and tracking of video objects for a content-based video indexing context , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[40]  Erik Blasch Book review [3C Vision: Cues, Content, and Channels (Cantoni, V. et al.; 2011)] , 2013 .

[41]  Li Bai,et al.  Robust infrared vehicle tracking across target pose change using L1 regularization , 2010, 2010 13th International Conference on Information Fusion.

[42]  Erik Blasch Modeling intent for a target tracking and identification scenario , 2004, SPIE Defense + Commercial Sensing.

[43]  Zheng Liu,et al.  Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision: A Comparative Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Genshe Chen,et al.  Wide-area motion imagery (WAMI) exploitation tools for enhanced situation awareness , 2012, 2012 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[45]  D. E. Brown,et al.  Context-based methods for track association , 2002, Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002. (IEEE Cat.No.02EX5997).

[46]  Gian Luca Foresti,et al.  Contexts, co-texts and situations in fusion domain , 2011, 14th International Conference on Information Fusion.

[47]  Dan Shen,et al.  Adaptive Markov Game Theoretic Data Fusion Approach for Cyber Network Defense , 2007, MILCOM 2007 - IEEE Military Communications Conference.

[48]  Emilio Maggio,et al.  Learning Scene Context for Multiple Object Tracking , 2009, IEEE Transactions on Image Processing.

[49]  Ana M. Sánchez,et al.  Video tracking improvement using context-based information , 2007, 2007 10th International Conference on Information Fusion.

[50]  Yufeng Zheng,et al.  Qualitative and quantitative comparisons of multispectral night vision colorization techniques , 2012, Optical Engineering.

[51]  Wolfgang Koch,et al.  On Bayesian tracking and data Fusion: A tutorial introduction with examples , 2010, IEEE Aerospace and Electronic Systems Magazine.

[52]  Lauro Snidaro,et al.  Fusing uncertain knowledge and evidence for maritime situational awareness via Markov Logic Networks , 2015, Inf. Fusion.

[53]  A. Soto,et al.  Design of an A-SMGCS prototype at Barajas airport: data fusion algorithms , 2005, 2005 7th International Conference on Information Fusion.

[54]  S. Plano,et al.  DFIG level 5 (user refinement) issues supporting situational assessment reasoning , 2005, 2005 7th International Conference on Information Fusion.

[55]  E. Blasch,et al.  Multiresolution EO/IR target tracking and identification , 2005, 2005 7th International Conference on Information Fusion.

[56]  Zhenchang Xing,et al.  Iterative context-aware feature location. , 2011, ICSE 2011.

[57]  Benjamin Pannetier,et al.  Extended and multiple target tracking: Evaluation of an hybridization solution , 2011, 14th International Conference on Information Fusion.

[58]  Erik Blasch,et al.  Cognitive fusion analysis based on context , 2004, SPIE Defense + Commercial Sensing.

[59]  Chun Yang,et al.  Optimal Placement of Heterogeneous Sensors for Targets with Gaussian Priors , 2013, IEEE Transactions on Aerospace and Electronic Systems.

[60]  Tarunraj Singh,et al.  Threat assessment using context-based tracking in a maritime environment , 2009, 2009 12th International Conference on Information Fusion.

[61]  W. Koch Information fusion aspects related to GMTI convoy tracking , 2002, Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002. (IEEE Cat.No.02EX5997).

[62]  John J. Salerno,et al.  Evaluating algorithmic techniques in supporting situation awareness , 2005, SPIE Defense + Commercial Sensing.

[63]  Ana M. Sánchez,et al.  A Context Model and Reasoning System to improve object tracking in complex scenarios , 2009, Expert Syst. Appl..

[64]  Tarak Gandhi,et al.  Pedestrian Protection Systems: Issues, Survey, and Challenges , 2007, IEEE Transactions on Intelligent Transportation Systems.

[65]  Erik Blasch,et al.  Performance Measures of Covariance and Information Matrices in Resource Management for Target State Estimation , 2012, IEEE Transactions on Aerospace and Electronic Systems.

[66]  Di Qiu,et al.  Underwater navigation using location-dependent signatures , 2012, 2012 IEEE Aerospace Conference.

[67]  K. S. Venkatesh,et al.  Formulation, detection and application of occlusion states (Oc-7) in the context of multiple object tracking , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[68]  Lang Hong,et al.  Identifying moving HRR signatures with an ATR belief data association filter , 2000, SPIE Defense + Commercial Sensing.

[69]  Erik Blasch,et al.  Context-driven moving vehicle detection in wide area motion imagery , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[70]  Li Bai,et al.  Multiple source data fusion via sparse representation for robust visual tracking , 2011, 14th International Conference on Information Fusion.

[71]  Miguel A. Patricio,et al.  Context-Based Reasoning Using Ontologies to Adapt Visual Tracking in Surveillance , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[72]  M.M. Trivedi,et al.  Performance evaluation of a vision based lane tracker designed for driver assistance systems , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[73]  Erik Blasch,et al.  GRoup IMM Tracking utilizing Track and Identification Fusion , 2001 .

[74]  Erik Blasch,et al.  High Level Information Fusion developments, issues, and grand challenges: Fusion 2010 panel discussion , 2010, 2010 13th International Conference on Information Fusion.

[75]  Gian Luca Foresti,et al.  Context in fusion: Some considerations in a JDL perspective , 2013, Proceedings of the 16th International Conference on Information Fusion.

[76]  Erik Blasch,et al.  Joint data management for MOVINT data-to-decision making , 2011, 14th International Conference on Information Fusion.

[77]  Duc Phu Chau,et al.  A multi-feature tracking algorithm enabling adaptation to context variations , 2011, ICDP.

[78]  Andrew Rice,et al.  Context-aided tracking with an adaptive hyperspectral sensor , 2011, 14th International Conference on Information Fusion.

[79]  John J. Salerno,et al.  Summary of human social, cultural, behavioral (HSCB) modeling for information fusion panel discussion , 2013, Defense, Security, and Sensing.

[80]  Erik Blasch,et al.  Feature-aided JBPDAF group tracking and classification using an IFFN sensor , 2002, SPIE Defense + Commercial Sensing.

[81]  I. Kadar,et al.  Resource management coordination with level 2/3 fusion issues and challenges [Panel Report] , 2008, IEEE Aerospace and Electronic Systems Magazine.

[82]  Václav Hlavác,et al.  Tracking with context as a semi-supervised learning and labeling problem , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[83]  Robert R. Tenney,et al.  Dynamic tactical targeting , 2004, SPIE Defense + Commercial Sensing.

[84]  Miguel A. Patricio,et al.  Ontology-based context representation and reasoning for object tracking and scene interpretation in video , 2011, Expert Syst. Appl..

[85]  C. Yang,et al.  Nonlinear constrained tracking of targets on roads , 2005, 2005 7th International Conference on Information Fusion.

[86]  P. Neskovic,et al.  Context-based tracking of object features , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[87]  Jesús García,et al.  Improving multiple-model context-aided tracking through an autocorrelation approach , 2012, 2012 15th International Conference on Information Fusion.

[88]  Jean-Pierre Le Cadre,et al.  Closed-form Posterior Cramér-Rao Bound for a Manoeuvring Target in the Bearings-Only Tracking Context Using Best-Fitting Gaussian Distribution , 2006, 2006 9th International Conference on Information Fusion.

[89]  Erik Blasch,et al.  Using maximum consistency context for multiple target association in wide area traffic scenes , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[90]  Erik Blasch,et al.  Decision-Level Fusion Performance Improvement From Enhanced HRR Radar Clutter Suppression , 2011, J. Adv. Inf. Fusion.

[91]  Li Bai,et al.  Real-Time Probabilistic Covariance Tracking With Efficient Model Update , 2012, IEEE Transactions on Image Processing.

[92]  Erik Blasch,et al.  Fusion of Tracks with Road Constraints , 2008, J. Adv. Inf. Fusion.

[93]  Yang Yang,et al.  Vision Target Tracker Based on Incremental Dictionary Learning and Global and Local Classification , 2013 .

[94]  Hyun-Taek Choi,et al.  Shape context based object recognition and tracking in structured underwater environment , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[95]  Chun Yang,et al.  Kalman Filtering with Nonlinear State Constraints , 2009 .

[96]  Erik Blasch,et al.  Encoding color information for visual tracking: Algorithms and benchmark , 2015, IEEE Transactions on Image Processing.

[97]  Michael Mertens,et al.  Ground Moving Target Tracking with context information and a refined sensor model , 2008, 2008 11th International Conference on Information Fusion.

[98]  Li Bai,et al.  Minimum error bounded efficient ℓ1 tracker with occlusion detection , 2011, CVPR 2011.

[99]  James Llinas,et al.  High Level Information Fusion (HLIF): Survey of models, issues, and grand challenges , 2012, IEEE Aerospace and Electronic Systems Magazine.

[100]  Erik Blasch,et al.  Three-dimensional receiver operating characteristic (ROC) trajectory concepts for the evaluation of target recognition algorithms faced with the unknown target detection problem , 1999, Defense, Security, and Sensing.

[101]  Erik Blasch,et al.  Revisiting the JDL model for information exploitation , 2013, Proceedings of the 16th International Conference on Information Fusion.

[102]  Miguel A. Patricio,et al.  Strategies and techniques for use and exploitation of Contextual Information in high-level fusion architectures , 2010, 2010 13th International Conference on Information Fusion.

[103]  Amiya Nayak,et al.  An Efficient Approach for Mobile Asset Tracking Using Contexts , 2012, IEEE Transactions on Parallel and Distributed Systems.

[104]  Roland Chapuis,et al.  Dealing with occlusions with multi targets tracking algorithms for the real road context , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[105]  Tarunraj Singh,et al.  Anomaly Detection using Context-Aided Target Tracking , 2011, J. Adv. Inf. Fusion.

[106]  Ali Borji,et al.  Adaptive object tracking by learning background context , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[107]  Lauro Snidaro,et al.  Integration of contextual information for tracking refinement , 2011, 14th International Conference on Information Fusion.

[108]  Genshe Chen,et al.  Multi-Pursuer Multi-Evader Pursuit-Evasion Games with Jamming Confrontation , 2007, J. Aerosp. Comput. Inf. Commun..

[109]  E. Blasch,et al.  Assembling a distributed fused information-based human-computer cognitive decision making tool , 2000, IEEE Aerospace and Electronic Systems Magazine.

[110]  Erik Blasch,et al.  Group tracking of occluded targets , 2001, SPIE Defense + Commercial Sensing.

[111]  Li Bai,et al.  Evaluation of visual tracking in extremely low frame rate wide area motion imagery , 2011, 14th International Conference on Information Fusion.

[112]  Erik Blasch,et al.  JDL level 5 fusion model: user refinement issues and applications in group tracking , 2002, SPIE Defense + Commercial Sensing.

[113]  I. Infantino,et al.  A System for Simultaneous People Tracking and Posture Recognition in the context of Human-Computer Interaction , 2005, EUROCON 2005 - The International Conference on "Computer as a Tool".

[114]  Erik Blasch,et al.  Automatic Association of Chats and Video Tracks for Activity Learning and Recognition in Aerial Video Surveillance , 2014, Sensors.

[115]  Joseph P. Olive,et al.  Formant tracking using context-dependent phonemic information , 2005, IEEE Transactions on Speech and Audio Processing.

[116]  Mubarak Shah,et al.  Motion and Appearance Contexts for Tracking and Re-Acquiring Targets in Aerial Videos , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.