Transfer Learning from an Auxiliary Discriminative Task for Unsupervised Anomaly Detection

Unsupervised anomaly detection from high dimensional data like mobility networks is a challenging task. Study of different approaches of feature engineering from such high dimensional data have been a focus of research in this field. This study aims to investigate the transferability of features learned by network classification to unsupervised anomaly detection. We propose use of an auxiliary classification task to extract features from unlabelled data by supervised learning, which can be used for unsupervised anomaly detection. We validate this approach by designing experiments to detect anomalies in mobility network data from New York and Taipei, and compare the results to traditional unsupervised feature learning approaches of PCA and autoencoders. We find that our feature learning approach yields best anomaly detection performance for both datasets, outperforming other studied approaches. This establishes the utility of this approach to feature engineering, which can be applied to other problems of similar nature.

[1]  Rikard Laxhammar,et al.  Anomaly detection for sea surveillance , 2008, 2008 11th International Conference on Information Fusion.

[2]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[3]  Stanislav Sobolevsky,et al.  Pattern and Anomaly Detection in Urban Temporal Networks , 2019, ArXiv.

[4]  Olivier Gibaru,et al.  CNN features are also great at unsupervised classification , 2017, ArXiv.

[5]  Yu Zheng,et al.  Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction , 2016, AAAI.

[6]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[7]  Carlo Ratti,et al.  Exploring Universal Patterns in Human Home-Work Commuting from Mobile Phone Data , 2013, PloS one.

[8]  Gregory Shakhnarovich,et al.  Learning Representations for Automatic Colorization , 2016, ECCV.

[9]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[10]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[11]  Carlo Ratti,et al.  Global multi-layer network of human mobility , 2016, Int. J. Geogr. Inf. Sci..

[12]  Paolo Favaro,et al.  Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles , 2016, ECCV.

[13]  Chuan Sheng Foo,et al.  Efficient GAN-Based Anomaly Detection , 2018, ArXiv.

[14]  Steve Harenberg,et al.  Anomaly detection in dynamic networks: a survey , 2015 .

[15]  Alexei A. Efros,et al.  Colorful Image Colorization , 2016, ECCV.

[16]  Clayton D. Scott,et al.  Robust kernel density estimation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Rex Britter,et al.  Predicting vehicular emissions in high spatial resolution using pervasively measured transportation data and microscopic emissions model , 2016 .

[18]  Yinhai Wang,et al.  Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecasting , 2018, IEEE Transactions on Intelligent Transportation Systems.

[19]  Zhiyong Cui,et al.  High-Order Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecasting , 2018, ArXiv.

[20]  Bo Zong,et al.  Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection , 2018, ICLR.

[21]  Alexei A. Efros,et al.  Unsupervised Visual Representation Learning by Context Prediction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Douglas A. Reynolds.  Gaussian Mixture Models , 2009, Encyclopedia of Biometrics.

[23]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[24]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[25]  Emilio Frazzoli,et al.  Revisiting Street Intersections Using Slot-Based Systems , 2016, PloS one.

[26]  Carlo Ratti,et al.  Geo-located Twitter as proxy for global mobility patterns , 2013, Cartography and geographic information science.

[27]  Carlo Ratti,et al.  Exploring human movements in Singapore: a comparative analysis based on mobile phone and taxicab usages , 2013, UrbComp '13.

[28]  Randy C. Paffenroth,et al.  Anomaly Detection with Robust Deep Autoencoders , 2017, KDD.

[29]  Fabrizio Angiulli,et al.  Anomaly Detection in Networks with Temporal Information , 2016, DS.

[30]  Paolo Santi,et al.  Supporting Information for Quantifying the Benefits of Vehicle Pooling with Shareability Networks Data Set and Pre-processing , 2022 .

[31]  Yong Wang,et al.  Learning Traffic as Images: A Deep Convolutional Neural Network for Large-Scale Transportation Network Speed Prediction , 2017, Sensors.

[32]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[33]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[34]  Georg Langs,et al.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery , 2017, IPMI.

[35]  Hans-Peter Kriegel,et al.  A survey on unsupervised outlier detection in high‐dimensional numerical data , 2012, Stat. Anal. Data Min..

[36]  Yu Cheng,et al.  Deep Structured Energy Based Models for Anomaly Detection , 2016, ICML.

[37]  Carlo Ratti,et al.  The impact of social segregation on human mobility in developing and industrialized regions , 2014, EPJ Data Science.

[38]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[39]  Carlo Ratti,et al.  A General Optimization Technique for High Quality Community Detection in Complex Networks , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[40]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[41]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[42]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[43]  Marc'Aurelio Ranzato,et al.  Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.