Group Deviation Detection Methods

Pointwise anomaly detection and change detection focus on the study of individual data instances; however, an emerging area of research involves groups or collections of observations. From applications of high-energy particle physics to health care collusion, group deviation detection techniques result in novel research discoveries, mitigation of risks, prevention of malicious collaborative activities, and other interesting explanatory insights. In particular, static group anomaly detection is the process of identifying groups that are not consistent with regular group patterns, while dynamic group change detection assesses significant differences in the state of a group over a period of time. Since both group anomaly detection and group change detection share fundamental ideas, this survey article provides a clearer and deeper understanding of group deviation detection research in static and dynamic situations.

[1]  T. Costigan,et al.  Bonferroni Inequalities and Intervals , 2005 .

[2]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[3]  Rainer Lienhart,et al.  Reliable Transition Detection in Videos: A Survey and Practitioner's Guide , 2001, Int. J. Image Graph..

[4]  Daniel J. Brass,et al.  Network Analysis in the Social Sciences , 2009, Science.

[5]  G. Box Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, II. Effects of Inequality of Variance and of Correlation Between Errors in the Two-Way Classification , 1954 .

[6]  Anne-Catherine Favre,et al.  Statistical inference in Lombard's smooth‐change model , 2011 .

[7]  Bernardete Ribeiro,et al.  Attribute Learning for Network Intrusion Detection , 2016, INNS Conference on Big Data.

[8]  Max Welling,et al.  Fast collapsed gibbs sampling for latent dirichlet allocation , 2008, KDD.

[9]  Vipin Kumar,et al.  The Challenges of Clustering High Dimensional Data , 2004 .

[10]  David J. Miller,et al.  Parsimonious Topic Models with Salient Word Discovery , 2014, IEEE Transactions on Knowledge and Data Engineering.

[11]  Andrew Y. Ng,et al.  Zero-Shot Learning Through Cross-Modal Transfer , 2013, NIPS.

[12]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[13]  Robert Lund,et al.  A Review and Comparison of Changepoint Detection Techniques for Climate Data , 2007 .

[14]  D. Siegmund,et al.  Sequential multi-sensor change-point detection , 2012, 2013 Information Theory and Applications Workshop (ITA).

[15]  F. Borgen,et al.  Uses of discriminant analysis following MANOVA: Multivariate statistics for multivariate purposes. , 1978 .

[16]  Christoph H. Lampert,et al.  Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Liang Xiong,et al.  On Learning from Collective Data , 2013 .

[18]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[19]  A. Zimek,et al.  On Using Class-Labels in Evaluation of Clusterings , 2010 .

[20]  Bernhard Schölkopf,et al.  One-Class Support Measure Machines for Group Anomaly Detection , 2013, UAI.

[21]  Ullas Gargi,et al.  Performance characterization of video-shot-change detection methods , 2000, IEEE Trans. Circuits Syst. Video Technol..

[22]  Arthur Zimek,et al.  On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study , 2016, Data Mining and Knowledge Discovery.

[23]  Shunzheng Yu,et al.  A Collaborative Intrusion Detection System against DDoS for SDN , 2016, IEICE Trans. Inf. Syst..

[24]  G. V. Kass An Exploratory Technique for Investigating Large Quantities of Categorical Data , 1980 .

[25]  David J. Miller,et al.  ATD: Anomalous Topic Discovery in High Dimensional Discrete Data , 2015, IEEE Transactions on Knowledge and Data Engineering.

[26]  Shaogang Gong,et al.  Unsupervised Domain Adaptation for Zero-Shot Learning , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[27]  Rand R. Wilcox,et al.  Comparing Two Independent Groups Via Multiple Quantiles , 1995 .

[28]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[29]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[30]  Bernt Schiele,et al.  Zero-Shot Learning — The Good, the Bad and the Ugly , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  J. Moors,et al.  A quantile alternative for kurtosis , 1988 .

[32]  Michalis Vazirgiannis,et al.  c ○ 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. On Clustering Validation Techniques , 2022 .

[33]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[34]  S. Kozlowski,et al.  Work Groups and Teams in Organizations , 2003 .

[35]  A. Moore,et al.  Wsare: What’s strange about recent events? , 2003, Journal of Urban Health.

[36]  Yutaka Matsuo,et al.  Earthquake shakes Twitter users: real-time event detection by social sensors , 2010, WWW '10.

[37]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[38]  Anil K. Jain Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[39]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[40]  Barnabás Póczos,et al.  Group Anomaly Detection using Flexible Genre Models , 2011, NIPS.

[41]  Edoardo M. Airoldi,et al.  Mixed Membership Stochastic Blockmodels , 2007, NIPS.

[42]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[43]  Rand R. Wilcox,et al.  Comparing two dependent groups via quantiles , 2012 .

[44]  Samy Bengio,et al.  Theme Topic Mixture Model: A Graphical Model for Document Representation , 2004 .

[45]  Nikos Karampatziakis,et al.  Online Discovery of Group Level Events in Time Series , 2014, SDM.

[46]  Tapani Raiko,et al.  Semi-supervised detection of collective anomalies with an application in high energy particle physics , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[47]  Claudio A. Agostini,et al.  Collusion On Private Health Insurance Coverage In Chile , 2011 .

[48]  S. Canu,et al.  Support Measure Data Description for group anomaly detection , 2015, KDD 2015.

[49]  Charu C. Aggarwal,et al.  Outlier Detection for Temporal Data: A Survey , 2014, IEEE Transactions on Knowledge and Data Engineering.

[50]  David V. Hinkley,et al.  On power transformations to symmetry , 1975 .

[51]  Ee-Peng Lim,et al.  Detecting Extreme Rank Anomalous Collections , 2012, SDM.

[52]  H. Akaike A new look at the statistical model identification , 1974 .

[53]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[54]  Ching-Yung Lin,et al.  A Survey on Social Media Anomaly Detection , 2016, SIGKDD Explor..

[55]  Ashbindu Singh,et al.  Review Article Digital change detection techniques using remotely-sensed data , 1989 .

[56]  Samuel J. Gershman,et al.  A Tutorial on Bayesian Nonparametric Models , 2011, 1106.2697.

[57]  Andrew W. Moore,et al.  Rule-based anomaly pattern detection for detecting disease outbreaks , 2002, AAAI/IAAI.

[58]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[59]  D. Darling The Kolmogorov-Smirnov, Cramer-von Mises Tests , 1957 .

[60]  C. Spearman The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[61]  Chong Wang,et al.  Stochastic variational inference , 2012, J. Mach. Learn. Res..

[62]  Junhui Wang,et al.  Detecting group review spam , 2011, WWW.

[63]  Rose Yu,et al.  GLAD: group anomaly detection in social media analysis , 2014, ACM Trans. Knowl. Discov. Data.

[64]  Christin Schäfer,et al.  Learning Intrusion Detection: Supervised or Unsupervised? , 2005, ICIAP.

[65]  Barnabás Póczos,et al.  Hierarchical Probabilistic Models for Group Anomaly Detection , 2011, AISTATS.

[66]  Douglas A. Reynolds,et al.  Speaker Verification Using Adapted Gaussian Mixture Models , 2000, Digit. Signal Process..