Finding Unusual Correlation Using Matrix Decompositions

One important aspect of terrorism detection is the ability to detect small-scale, local correlations against a background of large-scale, diffuse correlations. Several matrix decompositions transform correlation into other properties: for example, Singular Value Decomposition (SVD) transforms correlation into proximity, and SemiDiscrete Decomposition (SDD) transforms correlation into regions of increased density. Both matrix decompositions are effective at detecting local correlation in this setting, but they are much more effective when combined.

[1]  David B. Skillicorn,et al.  Semidiscrete Decomposition: A Bump Hunting Technique , 2002, AusDM.

[2]  Dianne P. O'Leary,et al.  Digital Image Compression by Outer Product Expansion , 1983, IEEE Trans. Commun..

[3]  David Jensen,et al.  Data Mining in Social Networks , 2002 .

[4]  David M. J. Tax,et al.  One-class classification , 2001 .

[5]  Sherry Marcus,et al.  Graph-based technologies for intelligence analysis , 2004, CACM.

[6]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[7]  D. O’Leary,et al.  Computation and Uses of the Semidiscrete Matrix Decomposition , 1999 .

[8]  M. Newman,et al.  Renormalization Group Analysis of the Small-World Network Model , 1999, cond-mat/9903357.

[9]  Gene H. Golub,et al.  Matrix computations , 1983 .

[10]  Raymond J. Mooney,et al.  Relational Data Mining with Inductive Logic Programming for Link Discovery , 2002 .

[11]  W. Baker,et al.  THE SOCIAL ORGANIZATION OF CONSPIRACY: ILLEGAL NETWORKS IN THE HEAVY ELECTRICAL EQUIPMENT INDUSTRY* , 1993 .

[12]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[13]  Caroline Haythornthwaite,et al.  Studying Online Social Networks , 2006, J. Comput. Mediat. Commun..

[14]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[15]  Tamara G. Kolda,et al.  A semidiscrete matrix decomposition for latent semantic indexing information retrieval , 1998, TOIS.

[16]  K. A. Taipale.  Data Mining and Domestic Security: Connecting the Dots to Make Sense of Data , 2004 .

[17]  Dimitris Achlioptas,et al.  Fast computation of low rank matrix approximations , 2001, STOC '01.

[18]  Susan T. Dumais,et al.  Using Linear Algebra for Intelligent Information Retrieval , 1995, SIAM Rev..

[19]  David D. Jensen,et al.  Information awareness: a prospective technical assessment , 2003, KDD '03.

[20]  Valdis E. Krebs,et al.  Mapping Networks of Terrorist Cells , 2001 .