Abnormal events detection using deep neural networks: application to extreme sea surface temperature detection in the Red Sea

Abstract. We present a method based on deep learning for detecting and localizing abnormal/extreme events in sea surface temperature (SST) of the Red Sea images using training samples of normal events only. The method operates in two stages; the first one involves features extraction from each patch of the SST input image using the first two convolutional layers extracted from a pretrained convolutional neural network. In the second stage, two methods are used for training the model from the normal training data. The first method uses one-class support vector machine (1-SVM) classifier that allows a fast and robust abnormal detection in the presence of outliers in the training dataset. In the second method, a Gaussian model is defined on the Mahalanobis distances between all normal training data. Experimental tests are conducted on satellite-derived SST data of the Red Sea spanning for a period of 31 years (1985–2015). Our results suggest that the Gaussian model of Mahalanobis distances outperformed 1-SVM by providing better performance in terms of sensitivity and specificity.

[1]  G. Meehl,et al.  More Intense, More Frequent, and Longer Lasting Heat Waves in the 21st Century , 2004, Science.

[2]  C. Bradshaw,et al.  Seaweed Communities in Retreat from Ocean Warming , 2011, Current Biology.

[3]  P. W. Glynn,et al.  Coral reef bleaching: ecological perspectives , 1993, Coral Reefs.

[4]  Christophe Rosenberger,et al.  Abnormal events detection based on spatio-temporal co-occurences , 2009, CVPR.

[5]  Mohan M. Trivedi,et al.  Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Christopher Leckie,et al.  High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning , 2016, Pattern Recognit..

[7]  V. Vapnik.  Pattern recognition using generalized portrait method , 1963 .

[8]  Nikos Paragios,et al.  Scene modeling and change detection in dynamic scenes: A subspace approach , 2009, Comput. Vis. Image Underst..

[9]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[10]  Michal Irani,et al.  Detecting Irregularities in Images and in Video , 2005, ICCV.

[11]  Wei Shen,et al.  Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes , 2016, Signal Process. Image Commun..

[12]  Michael T Burrows,et al.  Longer and more frequent marine heatwaves over the past century , 2018, Nature Communications.

[13]  Hichem Snoussi,et al.  Histograms of Optical Flow Orientation for Visual Abnormal Events Detection , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[14]  Mubarak Shah,et al.  Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Taghi M. Khoshgoftaar,et al.  Deep learning applications and challenges in big data analytics , 2015, Journal of Big Data.

[16]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[17]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[18]  Stavros J. Perantonis,et al.  Detecting abnormal human behaviour using multiple cameras , 2009, Signal Process..

[19]  Ehud Rivlin,et al.  Robust Real-Time Unusual Event Detection using Multiple Fixed-Location Monitors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Alistair J. Hobday,et al.  A hierarchical approach to defining marine heatwaves , 2016 .

[21]  Andrei Zaharescu,et al.  Anomalous Behaviour Detection Using Spatiotemporal Oriented Energies, Subset Inclusion Histogram Comparison and Event-Driven Processing , 2010, ECCV.

[22]  Wei Li,et al.  Transferred Deep Learning for Anomaly Detection in Hyperspectral Imagery , 2017, IEEE Geoscience and Remote Sensing Letters.

[23]  C. Donlon,et al.  OSTIA : An operational, high resolution, real time, global sea surface temperature analysis system , 2007, OCEANS 2007 - Europe.

[24]  S. Hales,et al.  Climate change and human health: present and future risks , 2006, The Lancet.

[25]  C. Donlon,et al.  The Operational Sea Surface Temperature and Sea Ice Analysis (OSTIA) system , 2012 .

[26]  Venkatesh Saligrama,et al.  Motion segmentation and abnormal behavior detection via behavior clustering , 2008, 2008 15th IEEE International Conference on Image Processing.

[27]  Simone Calderara,et al.  Detecting anomalies in people's trajectories using spectral graph analysis , 2011, Comput. Vis. Image Underst..

[28]  Lichao Mou,et al.  Learning a Transferable Change Rule from a Recurrent Neural Network for Land Cover Change Detection , 2016, Remote. Sens..

[29]  Mubarak Shah,et al.  Learning human actions via information maximization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Thomas S. Huang,et al.  One-class SVM for learning in image retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[31]  Louis Kratz,et al.  Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models , 2009, CVPR.

[32]  David A. Clausi,et al.  Goal-based trajectory analysis for unusual behaviour detection in intelligent surveillance , 2011, Image Vis. Comput..

[33]  A. Timmermann,et al.  Increasing frequency of extreme El Niño events due to greenhouse warming , 2014 .

[34]  Aggelos K. Katsaggelos,et al.  Anomalous video event detection using spatiotemporal context , 2011 .

[35]  Hichem Snoussi,et al.  Abnormal event detection using convolutional neural networks and 1-class SVM classifier , 2017, ICDP.

[36]  E. Madin,et al.  Socio-economic and management implications of range-shifting species in marine systems , 2012 .

[37]  Gian Luca Foresti,et al.  Trajectory-Based Anomalous Event Detection , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[38]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[39]  David J. Smith,et al.  Thermal refugia against coral bleaching throughout the northern Red Sea , 2018, Global change biology.

[40]  Christophe Rosenberger,et al.  Abnormal events detection based on spatio-temporal co-occurences , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Samy Bengio,et al.  Semi-supervised adapted HMMs for unusual event detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).