Generalization of feature embeddings transferred from different video anomaly detection domains

Abstract Detecting anomalous activity in video surveillance often suffers from limited availability of training data. Transfer learning may close this gap, allowing to use existing annotated data from some source domain. However, analyzing the source feature space in terms of its potential for transfer of learning to another context is still to be investigated. This paper reports a study on video anomaly detection, focusing on the analysis of feature embeddings of pre-trained CNNs with the use of novel cross-domain generalization measures that allow to study how source features generalize for different target video domains. This generalization analysis represents not only a theoretical approach, can be useful in practice as a path to understand which datasets allow better transfer of knowledge. Our results confirm this, achieving better anomaly detectors for video frames and allowing analysis of transfer learning’s positive and negative aspects.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Xinghao Ding,et al.  Vehicle Type Recognition in Surveillance Images From Labeled Web-Nature Data Using Deep Transfer Learning , 2018, IEEE Transactions on Intelligent Transportation Systems.

[3]  Aggelos K. Katsaggelos,et al.  Detecting contextual anomalies of crowd motion in surveillance video , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[4]  Andrew Zisserman,et al.  Tabula rasa: Model transfer for object category detection , 2011, 2011 International Conference on Computer Vision.

[5]  Rodrigo Fernandes de Mello,et al.  Providing theoretical learning guarantees to Deep Learning Networks , 2017, ArXiv.

[6]  Xiaolin Hu,et al.  Recurrent convolutional neural network for object recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Meng Wang,et al.  Transferring a generic pedestrian detector towards specific scenes , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Moacir Antonelli Ponti,et al.  Unsupervised Representation Learning Using Convolutional and Stacked Auto-Encoders: A Domain and Cross-Domain Feature Space Analysis , 2018, 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI).

[9]  Rodrigo Fernandes de Mello,et al.  Machine Learning: A Practical Approach on the Statistical Learning Theory , 2018 .

[10]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[11]  Moacir Antonelli Ponti,et al.  Everything You Wanted to Know about Deep Learning for Computer Vision but Were Afraid to Ask , 2017, 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images Tutorials (SIBGRAPI-T).

[12]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[13]  Yan Yan,et al.  Multi-label learning based deep transfer neural network for facial attribute classification , 2018, Pattern Recognit..

[14]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[15]  Gavriel Salomon,et al.  T RANSFER OF LEARNING , 1992 .

[16]  Peng Hao,et al.  Transfer learning using computational intelligence: A survey , 2015, Knowl. Based Syst..

[17]  Nizar Bouguila,et al.  Proportional data modeling with hidden Markov models based on generalized Dirichlet and Beta-Liouville mixtures applied to anomaly detection in public areas , 2016, Pattern Recognit..

[18]  Feng Xiao,et al.  Network traffic classification based on transfer learning , 2018, Comput. Electr. Eng..

[19]  Jiwen Lu,et al.  Deep transfer metric learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Xiaodong Yang,et al.  Evaluation of Low-Level Features for Real-World Surveillance Event Detection , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[22]  Imran N. Junejo,et al.  Social network model for crowd anomaly detection and localization , 2017, Pattern Recognit..

[23]  Yan Wang,et al.  Dense crowd counting from still images with convolutional neural networks , 2016, J. Vis. Commun. Image Represent..

[24]  Nannan Li,et al.  Quaternion discrete cosine transformation signature analysis in crowd scenes for abnormal event detection , 2016, Neurocomputing.

[25]  R. Grossman,et al.  On the Line , 2008 .

[26]  Nicu Sebe,et al.  Learning Cross-Modal Deep Representations for Robust Pedestrian Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Jonghyun Choi,et al.  Learning Temporal Regularity in Video Sequences , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Thomas S. Huang,et al.  One-class SVM for learning in image retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[29]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[30]  Ghassem Tofighi,et al.  Deep Learning-based Pipeline to Recognize Alzheimer’s Disease using fMRI Data , 2016, bioRxiv.

[31]  Josef Kittler,et al.  Optical-flow features empirical mode decomposition for motion anomaly detection , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[32]  Barbara Caputo,et al.  Safety in numbers: Learning categories from few examples with multi model knowledge transfer , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Bernhard Schölkopf,et al.  Statistical Learning Theory: Models, Concepts, and Results , 2008, Inductive Logic.

[34]  Venkatesh Saligrama,et al.  Modeling background activity for behavior subtraction , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[35]  I. Jolliffe.  Principal Component Analysis and Factor Analysis , 1986 .

[36]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[37]  Nuno Vasconcelos,et al.  Anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[39]  Hariharan Ravishankar,et al.  Understanding the Mechanisms of Deep Transfer Learning for Medical Images , 2016, LABELS/DLMIA@MICCAI.

[40]  Martin D. Levine,et al.  An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions , 2013, Comput. Vis. Image Underst..

[41]  Ivor W. Tsang,et al.  Visual Event Recognition in Videos by Learning from Web Data , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  J. Koenderink.  Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[44]  Dan Wu,et al.  Multi-view representation learning for multi-view action recognition , 2017, J. Vis. Commun. Image Represent..

[45]  Sarayu Parimal,et al.  Automatic segmentation of left ventricle in cardiac cine MRI images based on deep learning , 2017, Medical Imaging.

[46]  Andrei Zaharescu,et al.  Anomalous Behaviour Detection Using Spatiotemporal Oriented Energies, Subset Inclusion Histogram Comparison and Event-Driven Processing , 2010, ECCV.

[47]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[48]  Xiao Li,et al.  Projected Transfer Sparse Coding for cross domain image representation , 2015, J. Vis. Commun. Image Represent..

[49]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[50]  Trevor Darrell,et al.  Simultaneous Deep Transfer Across Domains and Tasks , 2015, ICCV.

[51]  Jing Lv,et al.  Vehicle counting in crowded scenes with multi-channel and multi-task convolutional neural networks , 2017, J. Vis. Commun. Image Represent..

[52]  Daoqiang Zhang,et al.  Robust multi-label transfer feature learning for early diagnosis of Alzheimer’s disease , 2018, Brain Imaging and Behavior.