Constructing and Analyzing Uncertain Social Networks from Unstructured Textual Data

Social network analysis and link diagrams are popular tools among intelligence analysts for analyzing and understanding criminal and terrorist organizations. A bottleneck in the use of such techniques is the manual effort needed to create the network to analyze from available source information. We describe how text mining techniques can be used for extraction of named entities and the relations among them, in order to enable automatic construction of networks from unstructured text. Since the text mining techniques used, viz. algorithms for named entity recognition and relation extraction, are not perfect, we also describe a method for incorporating information about uncertainty when constructing the networks and when doing the social network analysis. The presented approach is applied on text documents describing terrorist activities in Indonesia.

[1]  H. Milward,et al.  Dark Networks as Problems , 2003 .

[2]  John Skvoretz,et al.  Node centrality in weighted networks: Generalizing degree and shortest paths , 2010, Soc. Networks.

[3]  Ralph Grishman,et al.  Discovering Relations among Named Entities from Large Corpora , 2004, ACL.

[4]  Vladimir Batagelj,et al.  Pajek - Analysis and Visualization of Large Networks , 2001, Graph Drawing Software.

[5]  Gueorgi Kossinets.  Effects of missing data in social networks , 2006, Soc. Networks.

[6]  Satoshi Sekine,et al.  A survey of named entity recognition and classification , 2007 .

[7]  Kevin Chen-Chuan Chang,et al.  Searching patterns for relation extraction over the web: rediscovering the pattern-relation duality , 2011, WSDM '11.

[8]  Oren Etzioni,et al.  Open Information Extraction from the Web , 2007, CACM.

[9]  M. Newman,et al.  Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  Oren Etzioni,et al.  The Tradeoffs Between Open and Traditional Relation Extraction , 2008, ACL.

[11]  Stanley Wasserman,et al.  Social Network Analysis: Methods and Applications , 1994 .

[12]  Richard F. Deckro,et al.  Extraction Distractions: A Comparison of Social Network Model Construction Methods , 2011, 2011 European Intelligence and Security Informatics Conference.

[13]  Christopher Ré,et al.  Managing Uncertainty in Social Networks , 2007, IEEE Data Eng. Bull..

[14]  Thomas W. Valente,et al.  The stability of centrality measures when networks are sampled , 2003, Soc. Networks.

[15]  Bo Zhang,et al.  StatSnowball: a statistical approach to extracting entity relationships , 2009, WWW '09.

[16]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[17]  Richard F. Deckro,et al.  Determining Critical Members of Layered Operational Terrorist Networks , 2009 .

[18]  Sihem Amer-Yahia,et al.  Challenges in Searching Online Communities , 2007, IEEE Data Eng. Bull..

[19]  Johan Dahlin,et al.  A Method for Community Detection in Uncertain Networks , 2011, 2011 European Intelligence and Security Informatics Conference.

[20]  Terrill L. Frantz,et al.  Robustness of centrality measures under uncertainty: Examining the role of network topology , 2009, Comput. Math. Organ. Theory.

[21]  Vladimir Batagelj,et al.  Exploratory Social Network Analysis with Pajek , 2005 .

[22]  Kathleen M. Carley,et al.  On the robustness of centrality measures under conditions of imperfect data , 2006, Soc. Networks.

[23]  U. Brandes.  A faster algorithm for betweenness centrality , 2001 .

[24]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[25]  A. Vespignani,et al.  The architecture of complex weighted networks. , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[26]  L. Freeman.  Centrality in social networks conceptual clarification , 1978 .

[27]  Mark A. Przybocki,et al.  The Automatic Content Extraction (ACE) Program – Tasks, Data, and Evaluation , 2004, LREC.

[28]  Jian Su,et al.  Exploring Various Knowledge in Relation Extraction , 2005, ACL.

[29]  Ronen Feldman,et al.  Book Reviews: The Text Mining Handbook: Advanced Approaches to Analyzing Unstructured Data by Ronen Feldman and James Sanger , 2008, CL.

[30]  Mathieu Bastian,et al.  Gephi: An Open Source Software for Exploring and Manipulating Networks , 2009, ICWSM.

[31]  M E J Newman,et al.  Modularity and community structure in networks. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[32]  Kathleen McKeown,et al.  Extracting Social Networks from Literary Fiction , 2010, ACL.

[33]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[34]  M. Newman.  Analysis of weighted networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[35]  Johan Dahlin,et al.  Community Detection in Imperfect Networks , 2011 .

[36]  Bin Zhu,et al.  Visualizing social network concepts , 2010, Decis. Support Syst..

[37]  Dmitry Zelenko,et al.  Kernel methods for relation extraction , 2003 .

[38]  Ronald R. Yager.  Intelligent social network analysis using granular computing , 2008 .

[39]  Ivan P. Fellegi,et al.  A Theory for Record Linkage , 1969 .

[40]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.