Space-based Global Maritime Surveillance. Part II: Artificial Intelligence and Data Fusion Techniques

Maritime surveillance (MS) is of paramount importance for search and rescue operations, fishery monitoring, pollution control, law enforcement, migration monitoring, and national security policies. Since ground-based radars and automatic identification system (AIS) do not always provide a comprehensive and seamless coverage of the entire maritime domain, the use of space-based sensors is crucial to complement them. We reviewed space-based technologies for MS in the first part of this work, titled "Space-based Global Maritime Surveillance. Part I: Satellite Technologies" [1]. However, future MS systems combining multiple terrestrial and space-based sensors with additional information sources will require dedicated artificial intelligence and data fusion techniques for the processing of raw satellite images and fuse heterogeneous information. The second part of our work focuses on the most promising artificial intelligence and data fusion techniques for MS using space-based sensors.

[1]  Ryan Riddolls,et al.  A review of high frequency surface wave radar for detection and tracking of ships , 2010 .

[2]  Giorgio Battistelli,et al.  Joint attack detection and secure state estimation of cyber‐physical systems , 2016, International Journal of Robust and Nonlinear Control.

[3]  Anthony M. PONSFORD,et al.  A review of high frequency surface wave radar for detection and tracking of ships , 2010, Turkish Journal of Electrical Engineering and Computer Sciences.

[4]  Paolo Braca,et al.  Unsupervised extraction of maritime patterns of life from Automatic Identification System data , 2019, OCEANS 2019 - Marseille.

[5]  VarvarigouTheodora,et al.  Employing traditional machine learning algorithms for big data streams analysis , 2017 .

[6]  Konstantinos Tserpes,et al.  Employing traditional machine learning algorithms for big data streams analysis: The case of object trajectory prediction , 2016, J. Syst. Softw..

[7]  Giovanni De Magistris,et al.  Underwater Tracking Based on the Sum-Product Algorithm Enhanced by a Neural Network Detections Classifier , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Paolo Braca,et al.  Online Estimation of Unknown Parameters in Multisensor-Multitarget Tracking: a Belief Propagation Approach , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[9]  F. Hlawatsch,et al.  Classification-Aided Multitarget Tracking Using the Sum-Product Algorithm , 2020, IEEE Signal Processing Letters.

[10]  Moe Z. Win,et al.  Bayesian information fusion and multitarget tracking for maritime situational awareness , 2020, IET Radar, Sonar & Navigation.

[11]  M. Vespe,et al.  Unsupervised learning of maritime traffic patterns for anomaly detection , 2012 .

[12]  Antonio Iodice,et al.  Semantic Segmentation using Deep Learning: A case of study in Albufera Park, Valencia , 2019, 2019 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor).

[13]  Mark R. Morelande,et al.  Statistical analysis of motion patterns in AIS Data: Anomaly detection and motion prediction , 2008, 2008 11th International Conference on Information Fusion.

[14]  Guillaume Hajduch,et al.  A Multi-Task Deep Learning Architecture for Maritime Surveillance Using AIS Data Streams , 2018, 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA).

[15]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[16]  Paolo Braca,et al.  Belief Propagation Based AIS/Radar Data Fusion for Multi - Target Tracking , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[17]  Chee-Yee Chong,et al.  Forty Years of Multiple Hypothesis Tracking - A Review of Key Developments , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[18]  Zhe Xiao,et al.  Maritime Traffic Probabilistic Forecasting Based on Vessels’ Waterway Patterns and Motion Behaviors , 2017, IEEE Transactions on Intelligent Transportation Systems.

[19]  Paolo Braca,et al.  Maritime Surveillance Using Multiple High-Frequency Surface-Wave Radars , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Leto Peel,et al.  Maritime anomaly detection using Gaussian Process active learning , 2012, 2012 15th International Conference on Information Fusion.

[21]  Richard O. Lane,et al.  Maritime anomaly detection and threat assessment , 2010, 2010 13th International Conference on Information Fusion.

[22]  Paolo Braca,et al.  Modeling vessel kinematics using a stochastic mean-reverting process for long-term prediction , 2016, IEEE Transactions on Aerospace and Electronic Systems.

[23]  Michele Vespe,et al.  Vessel Pattern Knowledge Discovery from AIS Data: A Framework for Anomaly Detection and Route Prediction , 2013, Entropy.

[24]  Nicola Forti,et al.  Hybrid Bernoulli Filtering for Detection and Tracking of Anomalous Path Deviations , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[25]  Moe Z. Win,et al.  Message Passing Algorithms for Scalable Multitarget Tracking , 2018, Proceedings of the IEEE.

[26]  Simone Marinai,et al.  Historical Handwritten Document Segmentation by Using a Weighted Loss , 2018, ANNPR.

[27]  Thiagalingam Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation , 2001 .

[28]  Paolo Braca,et al.  Self-Tuning Algorithms for Multisensor-Multitarget Tracking Using Belief Propagation , 2019, IEEE Transactions on Signal Processing.

[29]  R. Mahler.  Multitarget Bayes filtering via first-order multitarget moments , 2003 .

[30]  Luca Cazzanti,et al.  Automated port traffic statistics: From raw data to visualisation , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[31]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[32]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[33]  Paolo Braca,et al.  Prediction oof Vessel Trajectories From AIS Data Via Sequence-To-Sequence Recurrent Neural Networks , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Paolo Braca,et al.  Validation of the Ornstein-Uhlenbeck route propagation model in the Mediterranean Sea , 2015, OCEANS 2015 - Genova.

[36]  Yu Huang,et al.  Trajectory compression-guided visualization of spatio-temporal AIS vessel density , 2016, 2016 8th International Conference on Wireless Communications & Signal Processing (WCSP).

[37]  Moe Z. Win,et al.  Heterogeneous Information Fusion for Multitarget Tracking Using the Sum-product Algorithm , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[38]  Peter Willett,et al.  Anomaly Detection and Tracking Based on Mean–Reverting Processes with Unknown Parameters , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Antonio Iodice,et al.  Space-based Global Maritime Surveillance. Part I: Satellite Technologies , 2020 .

[40]  Paolo Braca,et al.  Consistent Estimation of Randomly Sampled Ornstein–Uhlenbeck Process Long-Run Mean for Long-Term Target State Prediction , 2016, IEEE Signal Processing Letters.

[41]  Paolo Braca,et al.  Knowledge-Based Multitarget Ship Tracking for HF Surface Wave Radar Systems , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Paolo Braca,et al.  Multiple Ornstein–Uhlenbeck Processes for Maritime Traffic Graph Representation , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[43]  R. Mahler,et al.  PHD filters of higher order in target number , 2006, IEEE Transactions on Aerospace and Electronic Systems.

[44]  Marios Vodas,et al.  A Distributed Spatial Method for Modeling Maritime Routes , 2020, IEEE Access.

[45]  Paolo Braca,et al.  A Scalable Algorithm for Tracking an Unknown Number of Targets Using Multiple Sensors , 2016, IEEE Transactions on Signal Processing.

[46]  Allen M. Waxman,et al.  Associative Learning of Vessel Motion Patterns for Maritime Situation Awareness , 2006, 2006 9th International Conference on Information Fusion.

[47]  Ba-Ngu Vo,et al.  Analytic Implementations of the Cardinalized Probability Hypothesis Density Filter , 2007, IEEE Transactions on Signal Processing.

[48]  Paolo Braca,et al.  Maritime Anomaly Detection Based on Mean-Reverting Stochastic Processes Applied to a Real-World Scenario , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[49]  Roberto Cipolla,et al.  Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding , 2015, BMVC.

[50]  Ronald P. S. Mahler,et al.  Statistical Multisource-Multitarget Information Fusion , 2007 .

[51]  Edward H. Adelson,et al.  PYRAMID METHODS IN IMAGE PROCESSING. , 1984 .

[52]  Paolo Braca,et al.  Scalable distributed change detection and its application to maritime traffic , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[53]  Paolo Braca,et al.  Detecting Anomalous Deviations From Standard Maritime Routes Using the Ornstein–Uhlenbeck Process , 2018, IEEE Transactions on Signal Processing.

[54]  Liye Zhang,et al.  Traffic Pattern Mining and Forecasting Technologies in Maritime Traffic Service Networks: A Comprehensive Survey , 2020, IEEE Transactions on Intelligent Transportation Systems.

[55]  Ba-Ngu Vo,et al.  The Cardinality Balanced Multi-Target Multi-Bernoulli Filter and Its Implementations , 2009, IEEE Transactions on Signal Processing.

[56]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.