Analyzing Social Media through Big Data using InfoSphere BigInsights and Apache Flume

Abstract Social Media provides organizations ability to survey feelings towards the contents and events associated to them in real time. Moreover, the first demarche of the sentiment analysis is the pre-processing of data collected from Social Media. Most of existing research works that deals with social media analysis based on extracting new features related to sentiment. This paper presents the usage of Twitter in a number of proposed subjects, which is the largest social networking website where Twitter data is in increasing at higher rates every day that considers it as Big Data Source. Then, describing in detail the way in which Big data technology, such as, InfoSphere BigInsights enables processing of this data, which are primarily collected from social networks by Apache Flume and stored in Hadoop storage. In addition, we have investigated a Big Data platform for collecting social media data based on Apache Flume and analyzing this data using InfoSphere BigInsights. Moreover, our paper integrates the visualization of these analysis results using BigSheets. To that end, evaluation through analysis of results confirms that the proposed Big Data platform produces better results in terms of social media analysis.