Helping Intelligence Analysts Make Connections

Discovering latent connections between seemingly unconnected documents and constructing "stories" from scattered pieces of evidence are staple tasks in intelligence analysis. We have worked with government intelligence analysts to understand the strategies they use to make connections. Beyond techniques like clustering that aim to provide an initial broad summary of large document collections, an important goal of analysts in this domain is to assimilate and synthesize fine grained information from a smaller set of foraged documents. Further, analysts' domain expertise is crucial because it provides rich contextual background for making connections and thus the goal of KDD is to augment human discovery capabilities, not supplant it. We describe a visual analytics system we have built-- Analyst's Workspace (AW)--that integrates browsing tools with a storytelling algorithm in a large screen display environment. AW helps analysts systematically construct stories of desired fidelity from document collections and helps marshall evidence as longer stories are constructed.

[1]  P. Pirolli,et al.  The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis , 2007 .

[2]  Matthew Hurst,et al.  Deriving marketing intelligence from online discussion , 2005, KDD '05.

[3]  Ed Huai-hsin Chi,et al.  Entity Workspace: An Evidence File That Aids Memory, Inference, and Reading , 2006, ISI.

[4]  Catherine Plaisant,et al.  NetLens: Iterative Exploration of Content-Actor Network Data , 2006, 2006 IEEE Symposium On Visual Analytics Science And Technology.

[5]  Don R. Swanson,et al.  Complementary structures in disjoint science literatures , 1991, SIGIR '91.

[6]  Dafna Shahaf,et al.  Connecting the dots between news articles , 2011, IJCAI 2011.

[7]  Jan Gecsei,et al.  Path building in cellular partitioning networks , 1980, ISCA '80.

[8]  Robert Harper,et al.  Stories in GeoTime , 2007 .

[9]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[10]  Lucy T. Nowell,et al.  ThemeRiver: Visualizing Thematic Changes in Large Document Collections , 2002, IEEE Trans. Vis. Comput. Graph..

[11]  Edward A. Fox,et al.  Connecting topics in document collections with stepping stones and pathways , 2005, CIKM '05.

[12]  John Langford,et al.  Cover trees for nearest neighbor , 2006, ICML.

[13]  Padmini Srinivasan,et al.  Text mining: Generating hypotheses from MEDLINE , 2004, J. Assoc. Inf. Sci. Technol..

[14]  Concetto Spampinato,et al.  Discovering Genes-Diseases Associations From Specialized Literature Using the Grid , 2009, IEEE Transactions on Information Technology in Biomedicine.

[15]  Rohini K. Srihari,et al.  A Text Mining Model for Hypothesis Generation , 2007 .

[16]  Frank M. Shipman,et al.  Manipulating structured information in a visual workspace , 2002, UIST '02.

[17]  Frank M. Shipman,et al.  Formality Considered Harmful: Experiences, Emerging Themes, and Directions on the Use of Formal Representations in Interactive Systems , 1999, Computer Supported Cooperative Work (CSCW).

[18]  Snehasis Mukhopadhyay,et al.  Hypotheses Generation Pertaining to Ayurveda Using Automated Vocabulary Generation and Transitive Text Mining , 2009, 2009 International Conference on Network-Based Information Systems.

[19]  Naren Ramakrishnan,et al.  Reasoning about sets using redescription mining , 2005, KDD '05.

[20]  Snehasis Mukhopadhyay,et al.  Generating association graphs of non-cooccurring text objects using transitive methods , 2005, SAC '05.

[21]  John T. Stasko,et al.  Evaluating visual analytics systems for investigative analysis: Deriving design principles from a case study , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[22]  Christopher Andrews,et al.  Space to think: large high-resolution displays for sensemaking , 2010, CHI.

[23]  Alexander W. Skaburskis,et al.  The Sandbox for analysis: concepts and methods , 2006, CHI.

[24]  Naren Ramakrishnan,et al.  Algorithms for Storytelling , 2006, IEEE Transactions on Knowledge and Data Engineering.

[25]  Leonidas J. Guibas,et al.  Image webs: Computing and exploiting connectivity in image collections , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[27]  Himanshu Khurana,et al.  Palantir: a framework for collaborative incident response and investigation , 2009, IDtrust '09.

[28]  Krzysztof J. Cios,et al.  Text association mining with cross-sentence inference, structure-based document model and multi-relational text mining , 2009 .

[29]  Gary Sick,et al.  All Fall Down , 1985 .

[30]  Christos Faloutsos,et al.  Fast discovery of connection subgraphs , 2004, KDD.