ESCAPE Data Collection for Multi-Modal Data Fusion Research

Over the last decade there has been a technological explosion of advanced, digital, solid state, software controlled, and low size, weight, power and cost (SWaPC)sensors and payloads. The sensor advancement allowed engineering practitioners the ability to easily perform a variety of remote sensing operations and consider a wide range of modalities measuring the environment simultaneously. However, there has not been a common multi-modal data set to compare data fusion methods. This paper describes a multi-mode data set performed by the Air Force Research Laboratory, Information Directorate, to enable multi-modal signature data-fusion research. The Experiments, Scenarios, Concept of Operations, and Prototype Engineering (ESCAPE)collection brings together electro-optical, infrared, distributed passive radio-frequency, radar, acoustic and seismic data in a common scenario for the application of advanced fusion methods for aerospace systems. The paper details hardware, scenarios, and data collection specifics. Scenarios involved disparate moving emitting ground vehicles, challenging vehicle path patterns, and differing vehicle noise profiles. The purpose of the data collection, and the resulting data sets, is to engage the data fusion community in advanced upstream heterogeneous data analytics, design, and understanding.

[1]  Erik Blasch,et al.  Unmanned vehicles come of age: The DARPA grand challenge , 2006, Computer.

[2]  Olga Mendoza-Schrock,et al.  Video image registration evaluation for a layered sensing environment , 2009, Proceedings of the IEEE 2009 National Aerospace & Electronics Conference (NAECON).

[3]  Genshe Chen,et al.  A Real-Time High Performance Computation Architecture for Multiple Moving Target Tracking Based on Wide-Area Motion Imagery via Cloud and Graphic Processing Units , 2017, Sensors.

[4]  Andrew J. Newman,et al.  Upstream Data Fusion : History , Technical Overview , and Applications to Critical Challenges , 2013 .

[5]  Mohan M. Trivedi,et al.  Audiovisual Information Fusion in Human–Computer Interfaces and Intelligent Environments: A Survey , 2010, Proceedings of the IEEE.

[6]  Yu Hen Hu,et al.  Vehicle classification in distributed sensor networks , 2004, J. Parallel Distributed Comput..

[7]  Vijayan K. Asari,et al.  Moving Object Detection and Tracking in Wide Area Motion Imagery , 2014 .

[8]  B. Mulgrew,et al.  Multipolar SAR ATR: Experiments with the GTRI dataset , 2008, 2008 IEEE Radar Conference.

[9]  Erik Blasch,et al.  EIQIS: Toward an Event-Oriented Indexable and Queryable Intelligent Surveillance System , 2018, ArXiv.

[10]  Hubert Mara,et al.  Multivariate Data Analysis Using Persistence-Based Filtering and Topological Signatures , 2012, IEEE Transactions on Visualization and Computer Graphics.

[11]  Erik Blasch,et al.  Micro-Doppler radar classification of humans and animals in an operational environment , 2018, Expert Syst. Appl..

[12]  Peter Zulch,et al.  Dynamic sensor fusion using local topology , 2018, 2018 IEEE Aerospace Conference.

[13]  Roman Ilin,et al.  Extraction and classification of moving targets in multi-sensory MAMI-1 data collection , 2015, 2015 National Aerospace and Electronics Conference (NAECON).

[14]  Kyuchang Kang,et al.  ActionNet-VE Dataset: A Dataset for Describing Visual Events by Extending VIRAT Ground 2.0 , 2015, 2015 8th International Conference on Signal Processing, Image Processing and Pattern Recognition (SIP).

[15]  Erik Blasch,et al.  Encoding color information for visual tracking: Algorithms and benchmark , 2015, IEEE Transactions on Image Processing.

[16]  Joshua A. Levine,et al.  The Topology ToolKit , 2018, IEEE Transactions on Visualization and Computer Graphics.

[17]  Genshe Chen,et al.  Online single target tracking in WAMI: benchmark and evaluation , 2018, Multimedia Tools and Applications.

[18]  LeRoy A. Gorham,et al.  A challenge problem for SAR-based GMTI in urban environments , 2009, Defense + Commercial Sensing.

[19]  Larry S. Davis,et al.  AVSS 2011 demo session: A large-scale benchmark dataset for event recognition in surveillance video , 2011, AVSS.

[20]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Erik Blasch,et al.  Automatic Association of Chats and Video Tracks for Activity Learning and Recognition in Aerial Video Surveillance , 2014, Sensors.

[22]  Fang Liu,et al.  Upstream fusion of multiple sensing modalities using machine learning and topological analysis: An initial exploration , 2018, 2018 IEEE Aerospace Conference.

[23]  Erik Blasch,et al.  NAECON08 grand challenge entry using the belief filter in audio-video track and ID fusion , 2009, Proceedings of the IEEE 2009 National Aerospace & Electronics Conference (NAECON).

[24]  Holly Zelnio,et al.  Minor Area Motion Imagery (MAMI) Dismount Tower Data challenge problems , 2014, NAECON 2014 - IEEE National Aerospace and Electronics Conference.

[25]  Pierre Valin,et al.  DSmT applied to seismic and acoustic sensor fusion , 2011, Proceedings of the 2011 IEEE National Aerospace and Electronics Conference (NAECON).

[26]  Genshe Chen,et al.  Joint sparsity based heterogeneous data-level fusion for multi-target discovery , 2018, 2018 IEEE Aerospace Conference.

[27]  Genshe Chen,et al.  Manifold learning algorithms for sensor fusion of image and radio-frequency data , 2018, 2018 IEEE Aerospace Conference.

[28]  Udo Frese,et al.  Integrating generic sensor fusion algorithms with sound state representations through encapsulation of manifolds , 2011, Inf. Fusion.

[29]  Genshe Chen,et al.  A joint manifold leaning-based framework for heterogeneous upstream data fusion , 2018, Journal of Algorithms & Computational Technology.

[30]  Michael Lee Bryant,et al.  Standard SAR ATR evaluation experiments using the MSTAR public release data set , 1998, Defense, Security, and Sensing.

[31]  Erik Blasch,et al.  Moving object detection for vehicle tracking in Wide Area Motion Imagery using 4D filtering , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).