Space-Based Global Maritime Surveillance. Part II: Artificial Intelligence and Data Fusion Techniques

Maritime surveillance (MS) is of paramount importance for search and rescue operations, fishery monitoring, pollution control, law enforcement, migration monitoring, and national security policies. Since ground-based radars and automatic identification system (AIS) do not always provide a comprehensive and seamless coverage of the entire maritime domain, the use of space-based sensors is crucial to complement them. We reviewed space-based technologies for MS in the first part of this work, titled “Space-based Global Maritime Surveillance. Part I: Satellite Technologies.” However, MS systems combining multiple terrestrial and space-based sensors with additional information sources require dedicated artificial intelligence and data fusion techniques for processing raw satellite images and fusing heterogeneous information. The second part of our work focuses on some recent promising artificial intelligence and data fusion techniques for MS using space-based sensors.

[1]  A. Farina,et al.  Space-Based Global Maritime Surveillance. Part I: Satellite Technologies , 2020, IEEE Aerospace and Electronic Systems Magazine.

[2]  Moe Z. Win,et al.  Bayesian information fusion and multitarget tracking for maritime situational awareness , 2020, IET Radar, Sonar & Navigation.

[3]  Paolo Braca,et al.  Random Finite Set Tracking for Anomaly Detection in the Presence of Clutter , 2020, 2020 IEEE Radar Conference (RadarConf20).

[4]  F. Hlawatsch,et al.  Classification-Aided Multitarget Tracking Using the Sum-Product Algorithm , 2020, IEEE Signal Processing Letters.

[5]  Giovanni De Magistris,et al.  Underwater Tracking Based on the Sum-Product Algorithm Enhanced by a Neural Network Detections Classifier , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Liye Zhang,et al.  Traffic Pattern Mining and Forecasting Technologies in Maritime Traffic Service Networks: A Comprehensive Survey , 2020, IEEE Transactions on Intelligent Transportation Systems.

[7]  Paolo Braca,et al.  Prediction oof Vessel Trajectories From AIS Data Via Sequence-To-Sequence Recurrent Neural Networks , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Marios Vodas,et al.  A Distributed Spatial Method for Modeling Maritime Routes , 2020, IEEE Access.

[9]  Antonio Iodice,et al.  Semantic Segmentation using Deep Learning: A case of study in Albufera Park, Valencia , 2019, 2019 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor).

[10]  Paolo Braca,et al.  Self-Tuning Algorithms for Multisensor-Multitarget Tracking Using Belief Propagation , 2019, IEEE Transactions on Signal Processing.

[11]  Paolo Braca,et al.  Unsupervised extraction of maritime patterns of life from Automatic Identification System data , 2019, OCEANS 2019 - Marseille.

[12]  Peter Willett,et al.  Anomaly Detection and Tracking Based on Mean–Reverting Processes with Unknown Parameters , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Moe Z. Win,et al.  Heterogeneous Information Fusion for Multitarget Tracking Using the Sum-product Algorithm , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Paolo Braca,et al.  Detecting Anomalous Deviations From Standard Maritime Routes Using the Ornstein–Uhlenbeck Process , 2018, IEEE Transactions on Signal Processing.

[15]  Paolo Braca,et al.  Multiple Ornstein–Uhlenbeck Processes for Maritime Traffic Graph Representation , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[16]  Simone Marinai,et al.  Historical Handwritten Document Segmentation by Using a Weighted Loss , 2018, ANNPR.

[17]  Paolo Braca,et al.  Maritime Anomaly Detection Based on Mean-Reverting Stochastic Processes Applied to a Real-World Scenario , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[18]  Paolo Braca,et al.  Belief Propagation Based AIS/Radar Data Fusion for Multi - Target Tracking , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[19]  Nicola Forti,et al.  Hybrid Bernoulli Filtering for Detection and Tracking of Anomalous Path Deviations , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[20]  Paolo Braca,et al.  Online Estimation of Unknown Parameters in Multisensor-Multitarget Tracking: a Belief Propagation Approach , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[21]  Chee-Yee Chong,et al.  Forty Years of Multiple Hypothesis Tracking - A Review of Key Developments , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[22]  Guillaume Hajduch,et al.  A Multi-Task Deep Learning Architecture for Maritime Surveillance Using AIS Data Streams , 2018, 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA).

[23]  Moe Z. Win,et al.  Message Passing Algorithms for Scalable Multitarget Tracking , 2018, Proceedings of the IEEE.

[24]  Paolo Braca,et al.  Scalable distributed change detection and its application to maritime traffic , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[25]  Zhe Xiao,et al.  Maritime Traffic Probabilistic Forecasting Based on Vessels’ Waterway Patterns and Motion Behaviors , 2017, IEEE Transactions on Intelligent Transportation Systems.

[26]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[27]  Giorgio Battistelli,et al.  Joint attack detection and secure state estimation of cyber‐physical systems , 2016, International Journal of Robust and Nonlinear Control.

[28]  Luca Cazzanti,et al.  Automated port traffic statistics: From raw data to visualisation , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[29]  Paolo Braca,et al.  Modeling vessel kinematics using a stochastic mean-reverting process for long-term prediction , 2016, IEEE Transactions on Aerospace and Electronic Systems.

[30]  Yu Huang,et al.  Trajectory compression-guided visualization of spatio-temporal AIS vessel density , 2016, 2016 8th International Conference on Wireless Communications & Signal Processing (WCSP).

[31]  Paolo Braca,et al.  Consistent Estimation of Randomly Sampled Ornstein–Uhlenbeck Process Long-Run Mean for Long-Term Target State Prediction , 2016, IEEE Signal Processing Letters.

[32]  Konstantinos Tserpes,et al.  Employing traditional machine learning algorithms for big data streams analysis: The case of object trajectory prediction , 2016, J. Syst. Softw..

[33]  Paolo Braca,et al.  A Scalable Algorithm for Tracking an Unknown Number of Targets Using Multiple Sensors , 2016, IEEE Transactions on Signal Processing.

[34]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Roberto Cipolla,et al.  Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding , 2015, BMVC.

[36]  Anne-Laure Jousselme,et al.  Data-driven detection and context-based classification of maritime anomalies , 2015, 2015 18th International Conference on Information Fusion (Fusion).

[37]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[38]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[39]  Paolo Braca,et al.  Validation of the Ornstein-Uhlenbeck route propagation model in the Mediterranean Sea , 2015, OCEANS 2015 - Genova.

[40]  Paolo Braca,et al.  Knowledge-Based Multitarget Ship Tracking for HF Surface Wave Radar Systems , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Erik Blasch,et al.  Behavioral learning of vessel types with fuzzy-rough decision trees , 2014, 17th International Conference on Information Fusion (FUSION).

[42]  Paolo Braca,et al.  Maritime Surveillance Using Multiple High-Frequency Surface-Wave Radars , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Michele Vespe,et al.  Vessel Pattern Knowledge Discovery from AIS Data: A Framework for Anomaly Detection and Route Prediction , 2013, Entropy.

[44]  Leto Peel,et al.  Maritime anomaly detection using Gaussian Process active learning , 2012, 2012 15th International Conference on Information Fusion.

[45]  M. Vespe,et al.  Unsupervised learning of maritime traffic patterns for anomaly detection , 2012 .

[46]  Richard O. Lane,et al.  Maritime anomaly detection and threat assessment , 2010, 2010 13th International Conference on Information Fusion.

[47]  Ryan Riddolls,et al.  A review of high frequency surface wave radar for detection and tracking of ships , 2010 .

[48]  Anthony M. PONSFORD,et al.  A review of high frequency surface wave radar for detection and tracking of ships , 2010, Turkish Journal of Electrical Engineering and Computer Sciences.

[49]  Ba-Ngu Vo,et al.  The Cardinality Balanced Multi-Target Multi-Bernoulli Filter and Its Implementations , 2009, IEEE Transactions on Signal Processing.

[50]  Mark R. Morelande,et al.  Statistical analysis of motion patterns in AIS Data: Anomaly detection and motion prediction , 2008, 2008 11th International Conference on Information Fusion.

[51]  Peter Willett,et al.  Radar/AIS data fusion and SAR tasking for Maritime Surveillance , 2008, 2008 11th International Conference on Information Fusion.

[52]  Bradley J. Rhodes,et al.  Probabilistic associative learning of vessel motion patterns at multiple spatial scales for maritime situation awareness , 2007, 2007 10th International Conference on Information Fusion.

[53]  Ba-Ngu Vo,et al.  Analytic Implementations of the Cardinalized Probability Hypothesis Density Filter , 2007, IEEE Transactions on Signal Processing.

[54]  Ronald P. S. Mahler,et al.  Statistical Multisource-Multitarget Information Fusion , 2007 .

[55]  Allen M. Waxman,et al.  Associative Learning of Vessel Motion Patterns for Maritime Situation Awareness , 2006, 2006 9th International Conference on Information Fusion.

[56]  R. Mahler,et al.  PHD filters of higher order in target number , 2006, IEEE Transactions on Aerospace and Electronic Systems.

[57]  R. Mahler.  Multitarget Bayes filtering via first-order multitarget moments , 2003 .

[58]  Thiagalingam Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation , 2001 .

[59]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[60]  Edward H. Adelson,et al.  PYRAMID METHODS IN IMAGE PROCESSING. , 1984 .

[61]  Y. Bar-Shalom,et al.  Tracking And Data Fusion A Handbook Of Algorithms By | , 2017 .

[62]  Lauro Snidaro,et al.  Context-Enhanced Information Fusion , 2016, Advances in Computer Vision and Pattern Recognition.

[63]  D. Reid.  An algorithm for tracking multiple targets , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.