Maritime anomaly detection: A review

The surveillance of large sea areas normally requires the analysis of large volumes of heterogeneous, multidimensional and dynamic sensor data, in order to improve vessel traffic safety, maritime security and to protect the environment. Early detection of conflict situations at sea provides critical time to take appropriate action with, possibly before potential problems occur. In order to provide an overview of the state‐of‐the‐art of research carried out for the analysis of maritime data for situational awareness, this study presents a review of maritime anomaly detection. The found articles are categorized into four groups (a) data, (b) methods, (c) systems, and (d) user aspects. We present a comprehensive summary of the works found in each category, and finally, outline possible paths of investigation and challenges for maritime anomaly detection.

[1]  Allen M. Waxman,et al.  SeeCoast: persistent surveillance and automated scene understanding for ports and coastal areas , 2007, SPIE Defense + Commercial Sensing.

[2]  Nelson F. F. Ebecken,et al.  A SURVEY ON VIDEO DETECTION AND TRACKING OF MARITIME VESSELS , 2014 .

[3]  Etienne Martineau,et al.  Maritime Anomaly Detection: Domain Introduction and Review of Selected Literature , 2011 .

[4]  Maarten van Someren,et al.  Recognizing Vessel Movements from Historical Data , 2013, Situation Awareness with Systems of Systems.

[5]  Cynthia Rudin,et al.  Algorithms for interpretable machine learning , 2014, KDD.

[6]  B.J. Rhodes,et al.  SeeCoast: Automated Port Scene Understanding Facilitated by Normalcy Learning , 2006, MILCOM 2006 - 2006 IEEE Military Communications conference.

[7]  Dimitrios Zissis,et al.  A cloud based architecture capable of perceiving and predicting multiple vessel behaviour , 2015, Appl. Soft Comput..

[8]  D. Sperling,et al.  Four routes to better maritime governance , 2016, Nature.

[9]  Raffaele Grasso,et al.  Fusion of AIS, RADAR, and SAR data for maritime surveillance , 2007, SPIE Remote Sensing.

[10]  Melita Hadzagic,et al.  Information Mining Technologies to Enable Discovery of Actionable Intelligence to Facilitate Maritime Situational Awareness: I-MINE , 2013 .

[11]  Harm Greidanus,et al.  The declining impact of piracy on maritime transport in the Indian Ocean: Statistical analysis of 5-year vessel tracking data , 2015 .

[12]  David S. Ebert,et al.  A visual analytics process for maritime response, resource allocation and risk assessment , 2014, Inf. Vis..

[13]  Wen-Chih Peng,et al.  RouteMiner: Mining Ship Routes from a Massive Maritime Trajectories , 2014, 2014 IEEE 15th International Conference on Mobile Data Management.

[14]  John Stastny,et al.  Enhanced ship detection from overhead imagery , 2008, SPIE Defense + Commercial Sensing.

[15]  Camossi Elena,et al.  Development of a Web-Based Geographical Information System for Interactive Visualization and Analysis of Container Itineraries , 2014 .

[16]  Stan Matwin,et al.  Vessel route anomaly detection with Hadoop MapReduce , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[17]  Rikard Laxhammar,et al.  Anomaly detection for sea surveillance , 2008, 2008 11th International Conference on Information Fusion.

[18]  Aldo Napoli,et al.  Toward User-Centred Geovisual Analytics in Maritime Surveillance , 2013 .

[19]  Michele Vespe,et al.  Traffic knowledge discovery from AIS data , 2013, Proceedings of the 16th International Conference on Information Fusion.

[20]  Lily Rachmawati,et al.  Exploiting AIS Data for Intelligent Maritime Navigation: A Comprehensive Survey From Data to Methodology , 2016, IEEE Transactions on Intelligent Transportation Systems.

[21]  Jurgen Beyerer,et al.  Evaluation and comparison of anomaly detection algorithms in annotated datasets from the maritime domain , 2015, 2015 SAI Intelligent Systems Conference (IntelliSys).

[22]  Øystein Helleren,et al.  Operator and User Perspective of Fractionated AIS Satellite Systems , 2013 .

[23]  Paolo Braca,et al.  Detection of malicious AIS position spoofing by exploiting radar information , 2013, Proceedings of the 16th International Conference on Information Fusion.

[24]  Nikos Pelekis,et al.  Online event recognition from moving vessel trajectories , 2016, GeoInformatica.

[25]  Rikard Laxhammar,et al.  Anomaly Detection in Trajectory Data for Surveillance Applications , 2011 .

[26]  Branko Ristic,et al.  Detecting Anomalies from a Multitarget Tracking Output , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[27]  Kevin R. Keane Detecting motion anomalies , 2017, IWGS@SIGSPATIAL.

[28]  Aldo Napoli,et al.  High-Level Taxonomy of Geovisual Analytics Tasks for Maritime Surveillance , 2013 .

[29]  Kevin B. Korb,et al.  Anomaly detection in vessel tracks using Bayesian networks , 2014, Int. J. Approx. Reason..

[30]  Shibin Parameswaran,et al.  Evaluation of maritime object detection methods for full motion video applications using the PASCAL VOC Challenge framework , 2015, Electronic Imaging.

[31]  Maria Riveiro,et al.  Explanation Methods for Bayesian Networks : review and application to a maritime scenario , 2009 .

[32]  Ross Graham MDA Challenges for Operational Research and Analysis , 2009 .

[33]  Xavier Lerouvreur,et al.  Unsupervised extraction of knowledge from S-AIS data for maritime situational awareness , 2013, Proceedings of the 16th International Conference on Information Fusion.

[34]  Christoffer Brax,et al.  Anomaly detection in the surveillance domain , 2011 .

[35]  Milena Stróżyna,et al.  Architecture of Maritime Awareness System Supplied with External Information , 2016 .

[36]  Fernandez Arguedas Virginia,et al.  Unsupervised Maritime Pattern Analysis to Enhance Contextual Awareness , 2014 .

[37]  Stelios C. A. Thomopoulos,et al.  OCULUS SeaTM: integrated maritime surveillance platform , 2015, Defense + Security Symposium.

[38]  Tom Ziemke,et al.  Improving maritime anomaly detection and situation awareness through interactive visualization , 2008, 2008 11th International Conference on Information Fusion.

[39]  Steven C. Boraz Maritime Domain Awareness: Myths and Realities , 2009 .

[40]  Vladimir Radu Avram A spatio-temporal data representation framework with applications to anomaly detection in the maritime domain , 2012 .

[41]  Aaron Hunter,et al.  Belief modeling for maritime surveillance , 2009, 2009 12th International Conference on Information Fusion.

[42]  Göran Falkman,et al.  Interactive Visualization of Normal Behavioral Models and Expert Rules for Maritime Anomaly Detection , 2009, 2009 Sixth International Conference on Computer Graphics, Imaging and Visualization.

[43]  Daniel A. Keim,et al.  Human-centered machine learning through interactive visualization , 2016 .

[44]  Michele Vespe,et al.  Vessel Pattern Knowledge Discovery from AIS Data: A Framework for Anomaly Detection and Route Prediction , 2013, Entropy.

[45]  Allen M. Waxman,et al.  Associative Learning of Vessel Motion Patterns for Maritime Situation Awareness , 2006, 2006 9th International Conference on Information Fusion.

[46]  Michael Borth,et al.  Situation Awareness with Systems of Systems , 2013, Situation Awareness with Systems of Systems.

[47]  Aldo Napoli,et al.  ScanMaris : automatic detection of abnormal vessel behaviours , 2008 .

[48]  Claudio Ferrari,et al.  An Analysis of Shipping Agreements: The Cooperative Container Network , 2014 .

[49]  Valentina Dragos From Finding to Explaining: Information Retrieval to Support Maritime Anomaly Analysis , 2016, Int. J. Knowl. Syst. Sci..

[50]  Michael J Hanna Encounter Detection Using Visual Analytics to Improve Maritime Domain Awareness , 2015 .

[51]  Hans Wehn,et al.  Maritime Situation Analysis: A Multi-vessel Interaction and Anomaly Detection Framework , 2014, 2014 IEEE Joint Intelligence and Security Informatics Conference.

[52]  Anne-Laure Jousselme,et al.  Dissecting uncertainty-based fusion techniques for maritime anomaly detection , 2015, 2015 18th International Conference on Information Fusion (Fusion).

[53]  Dario Tarchi,et al.  AIS reception characterisation for AIS on/off anomaly detection , 2016, 2016 19th International Conference on Information Fusion (FUSION).

[54]  Lauro Snidaro,et al.  Fusing uncertain knowledge and evidence for maritime situational awareness via Markov Logic Networks , 2015, Inf. Fusion.

[55]  Harm Greidanus,et al.  SAR Image Quality Assessment and Indicators for Vessel and Oil Spill Detection , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[56]  Jesús García,et al.  Context-based Information Fusion: A survey and discussion , 2015, Inf. Fusion.

[57]  Tom Ziemke,et al.  VISAD: an interactive and visual analytical tool for the detection of behavioral anomalies in maritime traffic data , 2009, Defense + Commercial Sensing.

[58]  Fabio Mazzarella,et al.  Estimated Time of Arrival Using Historical Vessel Tracking Data , 2018, IEEE Transactions on Intelligent Transportation Systems.

[59]  Namje Park,et al.  A Requirement Analysis of Awareness-Based Vessel Traffic Service System for Maritime Safety , 2014 .

[60]  Dimitrova Tatyana,et al.  An Interactive Web-Based Geographical System in Assisting Location Matching Decisions , 2013 .

[61]  Paolo Braca,et al.  Context-enhanced vessel prediction based on Ornstein-Uhlenbeck processes using historical AIS traffic patterns: Real-world experimental results , 2014, 17th International Conference on Information Fusion (FUSION).

[62]  Giuliana Pallotta,et al.  Maritime Traffic Networks: From Historical Positioning Data to Unsupervised Maritime Traffic Monitoring , 2018, IEEE Transactions on Intelligent Transportation Systems.

[63]  Leonid Portnoy,et al.  Intrusion detection with unlabeled data using clustering , 2000 .

[64]  Michael Davenport Literature and Product Review of Visual Analytics for Maritime Awareness , 2009 .

[65]  Maria Riveiro,et al.  Evaluation of Normal Model Visualization for Anomaly Detection in Maritime Traffic , 2014, TIIS.

[66]  Gao Shu,et al.  Study of Automatic Anomalous Behaviour Detection Techniques for Maritime Vessels , 2017, Journal of Navigation.

[67]  Egils Sviestins,et al.  Rule-based situation assessment for sea surveillance , 2006, SPIE Defense + Commercial Sensing.

[68]  Visualizing Anomalies and Traffic Rules in a Maritime Setting Using Potential Fields , 2014 .

[69]  Valérie Lavigne Interactive Visualization Applications for Maritime Anomaly Detection and Analysis , 2014 .

[70]  A. Farina,et al.  Over the horizon maritime surveillance capability of DVB-T based Passive Radar , 2014, 2014 11th European Radar Conference.

[71]  B.J. Rhodes,et al.  Maritime situation monitoring and awareness using learning mechanisms , 2005, MILCOM 2005 - 2005 IEEE Military Communications Conference.

[72]  Daniele Nardi,et al.  Enhancing Automatic Maritime Surveillance Systems With Visual Information , 2017, IEEE Transactions on Intelligent Transportation Systems.

[73]  Richard O. Lane,et al.  Maritime anomaly detection and threat assessment , 2010, 2010 13th International Conference on Information Fusion.

[74]  Fabio Mazzarella,et al.  Discovering vessel activities at sea using AIS data: Mapping of fishing footprints , 2014, 17th International Conference on Information Fusion (FUSION).

[75]  Ashutosh Kumar Singh,et al.  Coastal surveillance in multi sensor environment: A design approach , 2016, 2016 3rd International Conference on Recent Advances in Information Technology (RAIT).

[76]  Cyril Ray,et al.  Spatio-temporal visualisation of outliers , 2011 .

[77]  Bengt Carlsson,et al.  Learning Maritime Traffic Rules Using Potential Fields , 2015, ICCL.

[78]  Josef Kittler,et al.  Maritime anomaly detection in ferry tracks , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[79]  Göran Falkman,et al.  Supporting the Analytical Reasoning Process in Maritime Anomaly Detection: Evaluation and Experimental Design , 2010, 2010 14th International Conference Information Visualisation.

[80]  Po-Ruey Lei,et al.  A framework for anomaly detection in maritime trajectory behavior , 2015, Knowledge and Information Systems.

[81]  Fabrizio Natale,et al.  Mapping Fishing Effort through AIS Data , 2015, PloS one.

[82]  Jarke J. van Wijk,et al.  Eurographics/ Ieee-vgtc Symposium on Visualization 2009 Visualization of Vessel Movements , 2022 .

[83]  Göran Falkman,et al.  Anomaly detection in sea traffic - A comparison of the Gaussian Mixture Model and the Kernel Density Estimator , 2009, 2009 12th International Conference on Information Fusion.

[84]  Jean Roy,et al.  Exploitation of maritime domain ontologies for anomaly detection and threat analysis , 2010, 2010 International WaterSide Security Conference.

[85]  B.J. Rhodes,et al.  Adaptive spatial scale for cognitively-inspired motion pattern learning & analysis algorithms for higher-level fusion and automated scene understanding , 2008, MILCOM 2008 - 2008 IEEE Military Communications Conference.

[86]  Daniel A. Keim,et al.  Mastering the Information Age - Solving Problems with Visual Analytics , 2010 .

[87]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[88]  Dario Tarchi,et al.  Radiolocation and tracking of automatic identification system signals for maritime situational awareness , 2015 .

[89]  M. Sciotti,et al.  Multi-sensor autonomous tracking for Maritime Surveillance , 2008, 2008 International Conference on Radar.

[90]  Aldo Napoli,et al.  Exploiting the Potential of the Future “Maritime Big Data” , 2016 .

[91]  Sangkyum Kim,et al.  ROAM: Rule- and Motif-Based Anomaly Detection in Massive Moving Object Data Sets , 2007, SDM.

[92]  Mark R. Morelande,et al.  Statistical analysis of motion patterns in AIS Data: Anomaly detection and motion prediction , 2008, 2008 11th International Conference on Information Fusion.

[93]  Stéphane Bressan,et al.  ASSIST: access controlled ship identification streams , 2011, Trans. Large Scale Data Knowl. Centered Syst..

[94]  Jochen Horstmann,et al.  Evaluation of an HF-radar ship detection and tracking algorithm by comparison to AIS and SAR data , 2010, 2010 International WaterSide Security Conference.

[95]  Natalie Fridman,et al.  KINGFISHER: Total Maritime Awareness System , 2017, AAMAS.

[96]  C. Ducruet Maritime Networks : Spatial structures and time dynamics , 2015 .

[97]  Aldo Napoli,et al.  Guiding the Controller in Geovisual Analytics to Improve Maritime Surveillance , 2013 .

[98]  Paulo Vicente,et al.  Maritime Domain Awareness , 2006 .

[99]  Fabrizio Cuccoli,et al.  Analysis of the radar coverage provided by a maritime Radar Network of Co-operative Vessels based on real AIS data , 2013, 2013 European Radar Conference.

[100]  Kevin B. Korb,et al.  Learning Abnormal Vessel Behaviour from AIS Data with Bayesian Networks at Two Time Scales , 2010 .

[101]  Ke Wang,et al.  Contextual verification for false alarm reduction in maritime anomaly detection , 2015, 2015 IEEE International Conference on Big Data (Big Data).

[102]  J.W.F. Wiersma,et al.  Assessing Vessel Traffic Service Operator Situation Awareness , 2010 .

[103]  David S. Ebert,et al.  TraSeer: A visual analytics tool for vessel movements in the coastal areas , 2017, 2017 IEEE International Symposium on Technologies for Homeland Security (HST).

[104]  Ewa Osekowska,et al.  Design and implementation of a maritime traffic modeling and anomaly detection method , 2014 .

[105]  Miriam A. M. Capretz,et al.  Contextual anomaly detection framework for big sensor data , 2015, Journal of Big Data.

[106]  Aungon Nag Radon False alarm reduction in maritime anomaly detection with contextual verification , 2015 .

[107]  Anders Brödje Hello, is there anybody out there – just nod if you can hear me , 2012 .

[108]  Fabio Mazzarella,et al.  Spatio-temporal data mining for maritime situational awareness , 2015, OCEANS 2015 - Genova.

[109]  Maria Riveiro,et al.  Visual analytics for maritime anomaly detection , 2011 .

[110]  Jenna Burrell,et al.  How the machine ‘thinks’: Understanding opacity in machine learning algorithms , 2016 .

[111]  Aldo Napoli,et al.  An enhanced spatial reasoning ontology for maritime anomaly detection , 2012, 2012 7th International Conference on System of Systems Engineering (SoSE).

[112]  Anders Holst,et al.  Incremental stream clustering and anomaly detection , 2008 .

[113]  Tom Ziemke,et al.  Reasoning about anomalies: a study of the analytical process of detecting and identifying anomalous behavior in maritime traffic data , 2009, Defense + Commercial Sensing.

[114]  Jos van Hillegersberg,et al.  Maritime pattern extraction and route reconstruction from incomplete AIS data , 2017, International Journal of Data Science and Analytics.

[115]  Paolo Braca,et al.  Maritime surveillance with multiple over-the-horizon HFSW radars: An overview of recent experimentation , 2015, IEEE Aerospace and Electronic Systems Magazine.

[116]  Bradley J. Rhodes,et al.  Probabilistic associative learning of vessel motion patterns at multiple spatial scales for maritime situation awareness , 2007, 2007 10th International Conference on Information Fusion.

[117]  A. Khatkhate,et al.  Modelling and system identification of an experimental apparatus for anomaly detection in mechanical systems , 2007 .

[118]  Paolo Braca,et al.  Application of the JPDA-UKF to HFSW radars for maritime situational awareness , 2012, 2012 15th International Conference on Information Fusion.

[119]  B. E. White,et al.  Case study: Maritime domain awareness , 2010, 2010 IEEE International Systems Conference.

[120]  Leto Peel,et al.  Maritime anomaly detection using Gaussian Process active learning , 2012, 2012 15th International Conference on Information Fusion.

[121]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[122]  Hans Hiemstra Defining a Balanced Maritime Safety and Security System , 2008 .

[123]  Paula Savioja,et al.  Challenges of developing the complex socio-technical system: realising the present, acknowledging the past, and envisaging the future of vessel traffic services. , 2007, Applied ergonomics.

[124]  Dario Tarchi,et al.  A novel anomaly detection approach to identify intentional AIS on-off switching , 2017, Expert Syst. Appl..

[125]  Metin Balci,et al.  Towards Global Maritime Domain Awareness - "Recent Developments and Challenges" , 2006, 2006 9th International Conference on Information Fusion.

[126]  A. Vandecasteele,et al.  Spatial ontologies for detecting abnormal maritime behaviour , 2012, 2012 Oceans - Yeosu.

[127]  Allen M. Waxman,et al.  Cognitively-Inspired Motion Pattern Learning & Analysis Algorithms for Higher-Level Fusion and Automated Scene Understanding , 2007, MILCOM 2007 - IEEE Military Communications Conference.

[128]  Jarke J. van Wijk,et al.  Visualization of Vessel Traffic , 2013, Situation Awareness with Systems of Systems.

[129]  Aldo Napoli,et al.  A semi-supervised learning framework based on spatio-temporal semantic events for maritime anomaly detection and behavior analysis , 2013 .

[131]  Elena Camossi,et al.  Visualization of Container Movements through a Web-Based Geographical Information System , 2013, 2013 European Intelligence and Security Informatics Conference.

[132]  Van Wimersma Greidanus Herman,et al.  Data Driven Contextual Knowledge from and for Maritime Situational Awareness , 2014 .

[133]  Yi Ming Dai,et al.  TACSA: A web-service based system for coastal surveillance and situational awareness , 2015, 2015 International Carnahan Conference on Security Technology (ICCST).

[134]  Wen-Chih Peng,et al.  ConflictFinder: Mining Maritime Traffic Conflict from Massive Ship Trajectories , 2017, 2017 18th IEEE International Conference on Mobile Data Management (MDM).

[135]  Josh Harguess,et al.  Sparsity-driven anomaly detection for ship detection and tracking in maritime video , 2015, Defense + Security Symposium.

[136]  Denis Gouin,et al.  Applicability of Visual Analytics to Defence and Security Operations , 2011 .

[137]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[138]  Maria Riveiro The importance of visualization and interaction in the anomaly detection process , 2013 .

[139]  Jean Roy,et al.  Rule-based expert system for maritime anomaly detection , 2010, Defense + Commercial Sensing.

[140]  Stan Matwin,et al.  Ship movement anomaly detection using specialized distance measures , 2015, 2015 18th International Conference on Information Fusion (Fusion).

[141]  Rikard Laxhammar,et al.  Conformal prediction for distribution-independent anomaly detection in streaming vessel data , 2010, StreamKDD '10.

[142]  James B. Kraiman,et al.  Automated anomaly detection processor , 2002, SPIE Defense + Commercial Sensing.

[143]  Alessandro Rossi,et al.  Sun-glint false alarm mitigation in a maritime scenario , 2014, Security and Defence.

[144]  Daniele Nardi,et al.  Integrated Visual Information for Maritime Surveillance , 2015 .

[145]  Alain Bouju,et al.  DeAIS project: Detection of AIS spoofing and resulting risks , 2015, OCEANS 2015 - Genova.

[146]  G. Galdorisi,et al.  Bridging the Policy and Technology Gap : A Process to Instantiate Maritime Domain Awareness , 2005, Proceedings of OCEANS 2005 MTS/IEEE.

[147]  Niklas Lavesson,et al.  Open data for anomaly detection in maritime surveillance , 2013, Expert Syst. Appl..

[148]  Tom Ziemke,et al.  Extracting rules from expert operators to support situation awareness in maritime surveillance , 2008, 2008 11th International Conference on Information Fusion.

[149]  Jean Roy,et al.  Anomaly detection in the maritime domain , 2008, SPIE Defense + Commercial Sensing.

[150]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[151]  Daniel Dahlmeier,et al.  sRADAR : A Complex Event Processing and Visual Analytics System for Maritime Intelligence , 2016 .

[152]  Luca Cazzanti,et al.  Mining maritime vessel traffic: Promises, challenges, techniques , 2015, OCEANS 2015 - Genova.

[153]  Per M. Gustavsson,et al.  Signature-based activity detection based on Bayesian networks acquired from expert knowledge , 2009, 2009 12th International Conference on Information Fusion.

[154]  Anne-Laure Jousselme,et al.  Data-driven detection and context-based classification of maritime anomalies , 2015, 2015 18th International Conference on Information Fusion (Fusion).

[155]  Véronique Malaisé,et al.  An integrated approach for visual analysis of a multisource moving objects knowledge base , 2010, Int. J. Geogr. Inf. Sci..

[156]  Léon J. M. Rothkrantz,et al.  A surveillance system of a military harbour using an automatic identification system , 2013, CompSysTech '13.

[157]  H. Griffiths,et al.  Passive coherent location radar systems. Part 1: performance prediction , 2005 .