On Optimal Quantization and its Effect on Anomaly Detection and Image Classification

This thesis presents the use of density estimation for performing data classification in different applications such as stream processing as well as image classification. The first half of this thesis presents a system that can process and analyze streaming data and extract the time frames that contain potential events of interest or anomalies without requiring any prior domain knowledge. The proposed method performs real time monitoring and mining of streaming data at multiple temporal scales simultaneously to maximize the probability of detection of anomalous events that span different lengths of time. The method does not assume the data segments containing anomalies belong to any particular distribution and therefore does not require prior domain knowledge. The system learns the evolution of normal behavior in streaming data and builds a model over time and uses it to determine whether the new incoming data fits that model. When analyzing streaming data, it is important for the algorithm to be fast with low computational complexity and therefore such aspects as well as the detection accuracy are studied and the results are presented. The algorithm is general and can be used for any type of streaming data. In the second half of this thesis, the feasibility of using density estimation in higher dimensions and in particular for visual descriptors is presented. A method for classifying images is proposed which uses density estimation to optimally quantize the feature space to generate a codebook used by a bag-of-features (BoF) image classifier. This thesis shows that the optimal smoothing calculation in density estimation can be used to systematically quantize the feature space to generate codebooks that can be used in image classification.

[1]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[2]  A. Hadi.  A Modification of a Method for the Detection of Outliers in Multivariate Samples , 1994 .

[3]  Kai Hwang,et al.  Frequent episode rules for Internet anomaly detection , 2004, Third IEEE International Symposium on Network Computing and Applications, 2004. (NCA 2004). Proceedings..

[4]  V. A. Epanechnikov.  Non-Parametric Estimation of a Multivariate Probability Density , 1969 .

[5]  S. Dolnicar.  A Review of Unquestioned Standards in Using Cluster Analysis for Data-Driven Market Segmentation , 2002 .

[6]  Salvatore J. Stolfo,et al.  Adaptive Intrusion Detection: A Data Mining Approach , 2000, Artificial Intelligence Review.

[7]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[8]  Lixin Fan,et al.  Categorizing Nine Visual Classes using Local Appearance Descriptors , 2004 .

[9]  R. Haddad,et al.  Multiresolution Signal Decomposition: Transforms, Subbands, and Wavelets , 1992 .

[10]  D. W. Scott,et al.  Multidimensional Density Estimation , 2005 .

[11]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[12]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Charu C. Aggarwal,et al.  A framework for diagnosing changes in evolving data streams , 2003, SIGMOD '03.

[14]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Chong-Wah Ngo,et al.  Towards optimal bag-of-features for object categorization and semantic video retrieval , 2007, CIVR '07.

[16]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[17]  Robert M. Gray,et al.  Vector quantization and density estimation , 1997, Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No.97TB100171).

[18]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[19]  D. W. Scott,et al.  Variable Kernel Density Estimation , 1992 .

[20]  I K Fodor,et al.  A Survey of Dimension Reduction Techniques , 2002 .

[21]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[22]  Cordelia Schmid,et al.  Affine-invariant local descriptors and neighborhood statistics for texture recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[23]  Eugene H. Spafford,et al.  An Application of Pattern Matching in Intrusion Detection , 1994 .

[24]  Tony Lindeberg,et al.  Detecting salient blob-like image structures and their scales with a scale-space primal sketch: A method for focus-of-attention , 1993, International Journal of Computer Vision.

[25]  Kwang-Ho Ro,et al.  Outlier detection for high-dimensional data , 2015 .

[26]  Shai Ben-David,et al.  Detecting Change in Data Streams , 2004, VLDB.

[27]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[28]  Peter Auer,et al.  Weak Hypotheses and Boosting for Generic Object Detection and Recognition , 2004, ECCV.

[29]  Alfonso Valdes,et al.  Adaptive, Model-Based Monitoring for Cyber Attack Detection , 2000, Recent Advances in Intrusion Detection.

[30]  P. Mermelstein,et al.  Distance measures for speech recognition, psychological and instrumental , 1976 .

[31]  Eamonn J. Keogh,et al.  Finding surprising patterns in a time series database in linear time and space , 2002, KDD.

[32]  G. Terrell.  The Maximal Smoothing Principle in Density Estimation , 1990 .

[33]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[34]  Ran Wolff,et al.  In-Network Outlier Detection in Wireless Sensor Networks , 2006, ICDCS.

[35]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[36]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[37]  Jitendra Malik,et al.  SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[38]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[39]  Nils L. Hjort.  On Frequency Polygons and Average Shifted Histograms in Higher Dimensions , 1986 .

[40]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[42]  Zubair A. Baig.  On the use of pattern matching for rapid anomaly detection in smart grid infrastructures , 2011, 2011 IEEE International Conference on Smart Grid Communications (SmartGridComm).

[43]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[44]  I. T. Jolliffe,et al.  Generalizations and Adaptations of Principal Component Analysis , 1986 .

[45]  Cordelia Schmid,et al.  Dimension Reduction and Classification Methods for Object Recognition in Vision , 2004 .

[46]  Giuseppe Sansonetti,et al.  Case-Based Anomaly Detection , 2007, ICCBR.

[47]  Andrew P. Witkin,et al.  Scale-Space Filtering , 1983, IJCAI.

[48]  Dimitrios Gunopulos,et al.  Online outlier detection in sensor data using non-parametric models , 2006, VLDB.

[49]  Qi Tian,et al.  Feature Extraction and Selection for Image Retrieval , 2000 .

[50]  Regunathan Radhakrishnan,et al.  Audio-visual event detection based on mining of semantic audio-visual labels , 2003, IS&T/SPIE Electronic Imaging.

[51]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[52]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, CVPR 2004.

[53]  Supavadee Aramvith,et al.  Dimensionality reduction of SIFT using PCA for object categorization , 2009, 2008 International Symposium on Intelligent Signal Processing and Communications Systems.

[54]  John W. Tukey,et al.  Exploratory Data Analysis , 1980, ACM SIGSPATIAL International Workshop on Advances in Geographic Information Systems.

[55]  Frédéric Jurie,et al.  Creating efficient codebooks for visual recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.