Protest Activity Detection and Perceived Violence Estimation from Social Media Images

We develop a novel visual model which can recognize protesters, describe their activities by visual attributes and estimate the level of perceived violence in an image. Studies of social media and protests use natural language processing to track how individuals use hashtags and links, often with a focus on those items' diffusion. These approaches, however, may not be effective in fully characterizing actual real-world protests (e.g., violent or peaceful) or estimating the demographics of participants (e.g., age, gender, and race) and their emotions. Our system characterizes protests along these dimensions. We have collected geotagged tweets and their images from 2013-2017 and analyzed multiple major protest events in that period. A multi-task convolutional neural network is employed in order to automatically classify the presence of protesters in an image and predict its visual attributes, perceived violence and exhibited emotions. We also release the UCLA Protest Image Dataset, our novel dataset of 40,764 images (11,659 protest images and hard negatives) with various annotations of visual attributes and sentiments. Using this dataset, we train our model and demonstrate its effectiveness. We also present experimental results from various analysis on geotagged image data in several prevalent protest events. Our dataset will be made accessible at https://www.sscnet.ucla.edu/comm/jjoo/mm-protest/.

[1]  Jisun An,et al.  #greysanatomy vs. #yankees: Demographics and Hashtag Use on Twitter , 2016, ICWSM.

[2]  Arnaldo de Albuquerque Araújo,et al.  Violence Detection in Video Using Spatio-Temporal Features , 2010, 2010 23rd SIBGRAPI Conference on Graphics, Patterns and Images.

[3]  Mahadev Satyanarayanan,et al.  OpenFace: A general-purpose face recognition library with mobile applications , 2016 .

[4]  Yiannis Kompatsiaris,et al.  Graph-Based Multimodal Clustering for Social Event Detection in Large Collections of Images , 2014, MMM.

[5]  Ingmar Weber,et al.  Characterizing the Demographics Behind the #BlackLivesMatter Movement , 2015, AAAI Spring Symposia.

[6]  Rahul Sukthankar,et al.  Violence Detection in Video Using Computer Vision Techniques , 2011, CAIP.

[7]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Zachary C. Steinert-Threlkeld,et al.  Structure, Agency, Hegemony, and Action: Ukrainian Nationalism in East Ukraine , 2016 .

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Zeynep Tufekci,et al.  Big Questions for Social Media Big Data: Representativeness, Validity and Other Methodological Pitfalls , 2014, ICWSM.

[11]  David A. Siegel,et al.  Coordination and security , 2015 .

[12]  Yamir Moreno,et al.  Broadcasters and Hidden Influentials in Online Protest Diffusion , 2012, ArXiv.

[13]  Tal Hassner,et al.  Violent flows: Real-time detection of violent crowd behavior , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[14]  R. A. Bradley,et al.  Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons , 1952 .

[15]  Song-Chun Zhu,et al.  Automated Facial Trait Judgment and Election Outcome Prediction: Social Dimensions of Face , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  Adriana Kovashka,et al.  WhittleSearch: Image search with relative attribute feedback , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Doug McAdam.  Recruitment to High-Risk Activism: The Case of Freedom Summer , 1986, American Journal of Sociology.

[18]  Yiannis Kompatsiaris,et al.  Social Event Detection at MediaEval 2012: Challenges, Dataset and Evaluation , 2012, MediaEval.

[19]  Karl-Dieter Opp,et al.  Rational Choice and Rebellious Collective Action , 1986, American Political Science Review.

[20]  Raquel Recuero,et al.  Taking tweets to the streets: A spatial analysis of the Vinegar Protests in Brazil , 2014, First Monday.

[21]  G. Tullock.  The paradox of revolution , 1971 .

[22]  Li-Yun Wang,et al.  Violence Detection in Movies , 2011, 2011 Eighth International Conference Computer Graphics, Imaging and Visualization.

[23]  Song-Chun Zhu,et al.  Visual Persuasion: Inferring Communicative Intents of Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Adriana Kovashka,et al.  Detecting Sexually Provocative Images , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[25]  Luc Van Gool,et al.  Visual interestingness in image sequences , 2013, MM '13.

[26]  Shaowen Wang,et al.  Mapping the global Twitter heartbeat: The geography of Twitter , 2013, First Monday.

[27]  Jiebo Luo,et al.  A Multifaceted Approach to Social Multimedia-Based Prediction of Elections , 2015, IEEE Transactions on Multimedia.

[28]  T. Kuran.  Sparks and prairie fires: A theory of unanticipated political revolution , 1989 .

[29]  M. Shamim Hossain,et al.  Social Event Classification via Boosted Multimodal Supervised Latent Dirichlet Allocation , 2015, ACM Trans. Multim. Comput. Commun. Appl..

[30]  S. Lohmann.  The Dynamics of Informational Cascades: The Monday Demonstrations in Leipzig, East Germany, 1989–91 , 1994, World Politics.

[31]  Jiebo Luo,et al.  Deciphering the 2016 U.S. Presidential Campaign in the Twitter Sphere: A Comparison of the Trumpists and Clintonists , 2016, ICWSM.

[32]  Zachary C. Steinert-Threlkeld.  Spontaneous Collective Action: Peripheral Mobilization During the Arab Spring , 2017, American Political Science Review.

[33]  Stuart Soroka,et al.  The Impact of News Photos on Support for Military Action , 2016 .

[34]  Guobin Yang,et al.  Achieving Emotions in Collective Action: Emotional Processes and Movement Mobilization in the 1989 Chinese Student Movement , 2000 .

[35]  Eric Gilbert,et al.  VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text , 2014, ICWSM.

[36]  Changsheng Xu,et al.  Cross-Domain Feature Learning in Multimedia , 2015, IEEE Transactions on Multimedia.

[37]  Alessandro Vespignani,et al.  Online social networks and offline protest , 2015, EPJ Data Science.

[38]  Joshua A. Tucker,et al.  The Critical Periphery in the Growth of Social Protests , 2015, PloS one.

[39]  Wendy Pearlman,et al.  Emotions and the Microfoundations of the Arab Uprisings , 2013, Perspectives on Politics.

[40]  Ruben Enikolopov,et al.  Social Media and Protest Participation: Evidence from Russia , 2016, Econometrica.

[41]  Chloe Kovacheff,et al.  Extreme Protest Tactics Reduce Popular Support for Social Movements , 2017 .

[42]  Jake Williams,et al.  Identifying violent protest activity with scalable machine learning ∗ , 2016 .

[43]  Jianxiong Xiao,et al.  What makes an image memorable , 2011 .

[44]  A. Little.  Communication Technology and Protest , 2016, The Journal of Politics.

[45]  Ebroul Izquierdo,et al.  Social event detection and retrieval in collaborative photo collections , 2012, ICMR '12.

[46]  Kristen Grauman,et al.  Relative attributes , 2011, 2011 International Conference on Computer Vision.

[47]  Yiannis Kompatsiaris,et al.  Social event detection using multimodal clustering and integrating supervisory signals , 2012, ICMR.

[48]  Rossano Schifanella,et al.  6 Seconds of Sound and Vision: Creativity in Micro-videos , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.