A joint manifold leaning-based framework for heterogeneous upstream data fusion

A joint manifold learning fusion (JMLF) approach is proposed for nonlinear or mixed sensor modalities with large streams of data. The multimodal sensor data are stacked to form joint manifolds, from which the embedded low intrinsic dimensionalities are discovered for moving targets. The intrinsic low dimensionalities are mapped to resolve the target locations. The JMLF framework is tested on digital imaging and remote sensing image generation scenes with mid-wave infrared (WMIR) data augmented with distributed radio-frequency (RF) Doppler data. Eight manifold learning methods are explored to train the system with the neighborhood preserving embedding showing promise for robust target tracking using video–radio-frequency fusion. The JMLF method shows a 93% improved accuracy as compared to a standard target tracking (e.g., Kalman-filter based) approach.

[1]  Scott D. Brown,et al.  Utilization of DIRSIG in support of real-time infrared scene generation , 2000, Defense, Security, and Sensing.

[2]  Andrea F. Cattoni,et al.  Video-radio fusion approach for target tracking in smart spaces , 2007, 2007 10th International Conference on Information Fusion.

[3]  Erik Blasch,et al.  Nonlinear target tracking for threat detection using RSSI and optical fusion , 2015, 2015 18th International Conference on Information Fusion (Fusion).

[4]  Genshe Chen,et al.  Information theoretic measures for performance evaluation and comparison , 2009, 2009 12th International Conference on Information Fusion.

[5]  Pramod K. Varshney,et al.  Distributed Detection and Decision Fusion with Applications to Wireless Sensor Networks , 2016 .

[6]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[7]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[8]  Genshe Chen,et al.  Context aided video-to-text information fusion , 2014, 17th International Conference on Information Fusion (FUSION).

[9]  C. Yang,et al.  Pose angular-aiding for maneuvering target tracking , 2005, 2005 7th International Conference on Information Fusion.

[10]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[11]  Li Bai,et al.  Efficient Minimum Error Bounded Particle Resampling L1 Tracker With Occlusion Detection , 2013, IEEE Transactions on Image Processing.

[12]  Erik Blasch,et al.  Random-point-based filters: analysis and comparison in target tracking , 2015, IEEE Transactions on Aerospace and Electronic Systems.

[13]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[14]  Zhonghai Wang,et al.  Video-based activity analysis using the L1 tracker on VIRAT data , 2013, 2013 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[15]  Amir Globerson,et al.  Metric Learning by Collapsing Classes , 2005, NIPS.

[16]  Zhonghai Wang,et al.  Pattern of life from WAMI objects tracking based on visual context-aware tracking and infusion network models , 2013, Defense, Security, and Sensing.

[17]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[18]  Hassaan Siddiqui.  UWB RTLS for Construction Equipment Localization: Experimental Performance Analysis and Fusion with Video Data , 2014 .

[19]  Carlo S. Regazzoni,et al.  Distributed Cognitive Sensor Network Approach for Surveillance Applications , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[20]  Erik Blasch,et al.  Decision-Level Fusion Performance Improvement From Enhanced HRR Radar Clutter Suppression , 2011, J. Adv. Inf. Fusion.

[21]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[22]  Chinmay Hegde,et al.  Joint Manifolds for Data Fusion , 2010, IEEE Transactions on Image Processing.

[23]  Erik Blasch,et al.  Comparison of stochastic integration filter with the Unscented Kalman filter for maneuvering targets , 2014, NAECON 2014 - IEEE National Aerospace and Electronics Conference.

[24]  Michael Mertens,et al.  Tracking and Data Fusion for Ground Surveillance , 2014 .

[25]  Mateen M. Rizki,et al.  Manifold and transfer subspace learning for cross-domain vehicle recognition in dynamic systems , 2015, 2015 18th International Conference on Information Fusion (Fusion).

[26]  Otman A. Basir,et al.  Radio-visual signal fusion for localization in cellular networks , 2010, 2010 IEEE Conference on Multisensor Fusion and Integration.

[27]  A. Neath.  Testing Statistical Hypotheses (3rd ed.). E. L. Lehmann and Joseph P. Romano , 2006 .

[28]  Roberto Pierdicca,et al.  Robust and affordable retail customer profiling by vision and radio beacon sensor fusion , 2016, Pattern Recognit. Lett..

[29]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[30]  Ann B. Lee,et al.  Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Erik Blasch,et al.  Automatic Association of Chats and Video Tracks for Activity Learning and Recognition in Aerial Video Surveillance , 2014, Sensors.

[32]  Erik Blasch,et al.  Data association through fusion of target track and identification sets , 2000, Proceedings of the Third International Conference on Information Fusion.

[33]  Sharath Pankanti,et al.  Video surveillance: past, present, and now the future [DSP Forum] , 2013, IEEE Signal Processing Magazine.

[34]  Pramod K. Varshney,et al.  Distributed Detection and Data Fusion , 1996 .

[35]  Andrew J. Newman,et al.  Upstream Data Fusion : History , Technical Overview , and Applications to Critical Challenges , 2013 .

[36]  Supun Samarasekera,et al.  Multi-modal sensor fusion algorithm for ubiquitous infrastructure-free localization in vision-impaired environments , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Genshe Chen,et al.  Image quality assessment for performance evaluation of image fusion , 2008, 2008 11th International Conference on Information Fusion.

[38]  Erik Blasch,et al.  Revisiting the JDL model for information exploitation , 2013, Proceedings of the 16th International Conference on Information Fusion.

[39]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[40]  Erik Blasch,et al.  Encoding color information for visual tracking: Algorithms and benchmark , 2015, IEEE Transactions on Image Processing.

[41]  Tat-Jun Chin,et al.  Improved wireless tracking using radio frequency and video sensors , 2013, Proceedings of the 16th International Conference on Information Fusion.

[42]  Genshe Chen,et al.  A low-cost, near-real-time two-UAS-based UWB emitter monitoring system , 2015, IEEE Aerospace and Electronic Systems Magazine.

[43]  Erik Blasch,et al.  Context-Enhanced Information Fusion: Boosting Real-World Performance with Domain Knowledge , 2016 .

[44]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Lauro Snidaro,et al.  Context-Enhanced Information Fusion , 2016, Advances in Computer Vision and Pattern Recognition.

[46]  Eloi Bosse,et al.  High-Level Information Fusion Management and System Design , 2012 .

[47]  L. Godara.  Application of antenna arrays to mobile communications. II. Beam-forming and direction-of-arrival considerations , 1997, Proc. IEEE.

[48]  P. Moral.  Nonlinear filtering : Interacting particle resolution , 1997 .

[49]  Erik Blasch,et al.  Mobile positioning via fusion of mixed signals of opportunity , 2014, IEEE Aerospace and Electronic Systems Magazine.

[50]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[51]  Chun Yang,et al.  Optimal Placement of Heterogeneous Sensors for Targets with Gaussian Priors , 2013, IEEE Transactions on Aerospace and Electronic Systems.