A Generic Architecture for a Social Network Monitoring and Analysis System

This paper describes the architecture and a partial implementation of a system designed for the monitoring and analysis of communities at social media sites. The main contribution of the paper is a novel system architecture that facilitates long-term monitoring of diverse social networks existing and emerging at various social media sites. It consists of three main modules, the crawler, the repository and the analyzer. The first module can be adapted to crawl different sites based on ontology describing the structure of the site. The repository stores the crawled and analyzed persistent data using efficient data structures. It can be implemented using special purpose graph databases and/or object-relational database. The analyzer hosts modules that can be used for various graph and multimedia contents analysis tasks. The results can be again stored to the repository, and so on. All modules can be run concurrently.

[1]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[2]  Alexander Lazovik,et al.  Mining Twitter in the Cloud: A Case Study , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[3]  Yoelle Maarek,et al.  The Shark-Search Algorithm. An Application: Tailored Web Site Mapping , 1998, Comput. Networks.

[4]  Michael J. Paul,et al.  Cross-Cultural Analysis of Blogs and Forums with Mixed-Collection Topic Models , 2009, EMNLP.

[5]  Kathleen de la Peña McCook,et al.  A place at the table : participating in community building , 2000 .

[6]  Prasenjit Mitra,et al.  Clustering-based incremental web crawling , 2010, TOIS.

[7]  Christophe G. Giraud-Carrier,et al.  Implicit affinity networks and social capital , 2009, Inf. Technol. Manag..

[8]  Jouni Markkula,et al.  An integrated identity verification system for mobile terminals , 2005, Inf. Manag. Comput. Security.

[9]  Alain Wegmann,et al.  Stakeholder discovery and classification based on systems science principles , 2001, Proceedings Second Asia-Pacific Conference on Quality Software.

[10]  Nicholas C. Wormald,et al.  Representing Small Group Evolution , 2009, 2009 International Conference on Computational Science and Engineering.

[11]  Donald Kossmann,et al.  AJAXSearch: crawling, indexing and searching web 2.0 applications , 2008, Proc. VLDB Endow..

[12]  G. A. Hillery.  Definitions of community : Areas of Agreement , 1955 .

[13]  Lefteris Kozanidis,et al.  An Ontology-Based Focused Crawler , 2008, NLDB.

[14]  Maurice Tchuente,et al.  Local Community Identification in Social Networks , 2012, Parallel Process. Lett..

[15]  Brian Nicholson,et al.  Conviviality of Internet social networks: An exploratory study of Internet campaigns in Iran , 2010, J. Inf. Technol..

[16]  Hsinchun Chen,et al.  Terrorism Informatics: Knowledge Management and Data Mining for Homeland Security , 2008 .

[17]  Charu C. Aggarwal,et al.  Social Network Data Analytics , 2011 .

[18]  A. Kaplan,et al.  Users of the world, unite! The challenges and opportunities of Social Media , 2010 .

[19]  James E. Tomayko,et al.  Software architecture-centric methods and agile development , 2006, IEEE Software.

[20]  Efstathios Stamatatos,et al.  A survey of modern authorship attribution methods , 2009, J. Assoc. Inf. Sci. Technol..

[21]  Pengpeng Zhao,et al.  Ontology-Based Focused Crawling of Deep Web Sources , 2007, KSEM.

[22]  P. Kollock.  The Economies of Online Cooperation: Gifts and Public Goods in Cyberspace , 1999 .

[23]  Ben Y. Zhao,et al.  Understanding latent interactions in online social networks , 2010, IMC '10.

[24]  Antony Stephen Reid Manstead,et al.  The Blackwell Encyclopedia of Social Psychology , 1996 .

[25]  Jari Veijalainen,et al.  Analysing the presence of school-shooting related communities at social media sites , 2010, Int. J. Multim. Intell. Secur..

[26]  Euripides G. M. Petrakis,et al.  Improving the performance of focused web crawlers , 2009, Data Knowl. Eng..

[27]  Randy Goebel,et al.  Local Community Identification in Social Networks , 2009, 2009 International Conference on Advances in Social Network Analysis and Mining.

[28]  Guido van Rossum,et al.  Python Programming Language , 2007, USENIX Annual Technical Conference.

[29]  Naizhou Zhang,et al.  Applying a Multi-Attribute Metrics Approach to Detect Contents of Blog Communities , 2008, 2008 4th International Conference on Wireless Communications, Networking and Mobile Computing.

[30]  Douglas R. Vogel,et al.  Virtual Community Informatics: A Review and Research Agenda , 2003 .

[31]  Jure Leskovec,et al.  Empirical comparison of algorithms for network community detection , 2010, WWW '10.

[32]  Uffe Kock Wiil,et al.  EWAS: Modeling Application for Early Detection of Terrorist Threats , 2010, From Sociology to Computing in Social Networks.

[33]  Christos Faloutsos,et al.  Parallel crawling for online social networks , 2007, WWW '07.

[34]  Oliver Günther,et al.  Why Participate in an Online Social Network? An Empirical Analysis , 2008, ECIS.

[35]  Timothy W. Finin,et al.  Why we twitter: understanding microblogging usage and communities , 2007, WebKDD/SNA-KDD '07.

[36]  Shane Warden,et al.  The art of agile development , 2007 .

[37]  Sheng-Yuan Yang.  Developing of an Ontological Focused-Crawler for Ubiquitous Services , 2008, 22nd International Conference on Advanced Information Networking and Applications - Workshops (aina workshops 2008).

[38]  Olfa Nasraoui,et al.  Profile-Based Focused Crawling for Social Media-Sharing Websites , 2008, 2008 20th IEEE International Conference on Tools with Artificial Intelligence.

[39]  Bo Yuan,et al.  An Improved Shark-Search Algorithm Based on Multi-information , 2007, Fourth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2007).

[40]  Arno Scharl,et al.  Distributed Web2.0 crawling for ontology evolution , 2007, 2007 2nd International Conference on Digital Information Management.

[41]  Sheng-Yuan Yang,et al.  An ontology-supported web focused-crawler for Java programs , 2010, 2010 3rd IEEE International Conference on Ubi-Media Computing.

[42]  Craig Ross,et al.  The Influence of Shyness on the Use of Facebook in an Undergraduate Sample , 2009, Cyberpsychology Behav. Soc. Netw..

[43]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[44]  Joe Marini,et al.  Document Object Model , 2002, Encyclopedia of GIS.