A document-based data model for large scale computational maritime situational awareness

Computational Maritime Situational Awareness (MSA) supports the maritime industry, governments, and international organizations with machine learning and big data techniques for analyzing vessel traffic data available through the Automatic Identification System (AIS). A critical challenge of scaling computational MSA to big data regimes is integrating the core learning algorithms with big data storage modes and data models. To address this challenge, we report results from our experimentation with MongoDB, a NoSQL document-based database which we test as a supporting platform for computational MSA. We experiment with a document model that avoids database joins when linking position and voyage AIS vessel information and allows tuning the database index and document sizes in response to the AIS data rate. We report results for the AIS data ingested and analyzed daily at the NATO Centre for Maritime Research and Experimentation (CMRE).

[1]  Michele Vespe,et al.  Vessel Pattern Knowledge Discovery from AIS Data: A Framework for Anomaly Detection and Route Prediction , 2013, Entropy.

[2]  Xavier Lerouvreur,et al.  Unsupervised extraction of knowledge from S-AIS data for maritime situational awareness , 2013, Proceedings of the 16th International Conference on Information Fusion.

[3]  B.J. Tetreault,et al.  Use of the Automatic Identification System (AIS) for maritime domain awareness (MDA) , 2005, Proceedings of OCEANS 2005 MTS/IEEE.

[4]  Yasuhiro Nakamura,et al.  Predicting Ship Behavior Navigating through Heavily Trafficked Fairways by Analyzing AIS Data on Apache HBase , 2013, 2013 First International Symposium on Computing and Networking.

[5]  Stan Matwin,et al.  Vessel route anomaly detection with Hadoop MapReduce , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[6]  Fuchun Sun,et al.  Vessel track information mining using AIS data , 2014, 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems (MFI).

[7]  Lars Linsen,et al.  Comprehensive Analysis of Automatic Identification System (AIS) Data in Regard to Vessel Movement Prediction , 2014 .

[8]  Robert J. Meijer,et al.  Sensor Data Storage Performance: SQL or NoSQL, Physical or Virtual , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[9]  Bradley J. Rhodes,et al.  Probabilistic prediction of vessel motion at multiple spatial scales for maritime situation awareness , 2008, 2008 11th International Conference on Information Fusion.

[10]  Fabio Mazzarella,et al.  Discovering vessel activities at sea using AIS data: Mapping of fishing footprints , 2014, 17th International Conference on Information Fusion (FUSION).

[11]  Bradley J. Rhodes,et al.  Probabilistic associative learning of vessel motion patterns at multiple spatial scales for maritime situation awareness , 2007, 2007 10th International Conference on Information Fusion.

[12]  Stan Matwin,et al.  Knowledge-based clustering of ship trajectories using density-based approach , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[13]  Hans-Peter Kriegel,et al.  Scalable Density-Based Distributed Clustering , 2004, PKDD.

[14]  Hans-Peter Kriegel,et al.  A Fast Parallel Clustering Algorithm for Large Spatial Databases , 1999, Data Mining and Knowledge Discovery.

[15]  Steven Horn,et al.  CMRE-FR-2014-017 Sensor data management to achieve information superiority in maritime situational awareness , 2014 .

[16]  Di Ma,et al.  MR-DBSCAN: An Efficient Parallel Density-Based Clustering Algorithm Using MapReduce , 2011, 2011 IEEE 17th International Conference on Parallel and Distributed Systems.

[17]  Paolo Braca,et al.  Estimating sensor performance and target population size with multiple sensors , 2012, 2012 15th International Conference on Information Fusion.

[18]  Mark R. Morelande,et al.  Statistical analysis of motion patterns in AIS Data: Anomaly detection and motion prediction , 2008, 2008 11th International Conference on Information Fusion.

[19]  Maarten van Someren,et al.  Machine learning for vessel trajectories using compression, alignments and domain knowledge , 2012, Expert Syst. Appl..

[20]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[21]  Branko Ristic,et al.  Detecting Anomalies from a Multitarget Tracking Output , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[22]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.