Joint user-interest and social-influence emotion prediction for individuals

Emotions are playing significant roles in daily life, making emotion prediction important. Since emotions are highly subjective for users, we focus on emotion prediction for individuals instead of for the masses. Existing works on emotion prediction for individuals either focus on media content or social-influence alone, which are incomprehensive in predicting users' emotions. In this paper, we design a joint user-interest and social-influence emotion prediction framework for individuals, in which user-interest in multimodal media content and social-influence in social relations are both considered. To address the issue that for different users, the impacts of these two factors are different, a probabilistic graphical model is proposed to combine these two factors together, where a set of parameters are used to measure their importance in influencing the user's emotions, and they are learnt from the user's historical behaviors. We conduct experiments using real social media network to verify our algorithm and evaluate its performance. The results demonstrate the effectiveness of our approach and show that our approach can substantially improve the emotion prediction performance.

[1]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[2]  Thomas Gottron,et al.  Bad news travel fast: a content-based analysis of interestingness on Twitter , 2011, WebSci '11.

[3]  Yiran Chen,et al.  Quantitative Study of Individual Emotional States in Social Networks , 2012, IEEE Transactions on Affective Computing.

[4]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[5]  Reiner Lenz,et al.  Color Emotions for Image Classification and Retrieval , 2008, CGIV/MCS.

[6]  Dan Frankowski,et al.  Collaborative Filtering Recommender Systems , 2007, The Adaptive Web.

[7]  Lifeng Sun,et al.  Who should share what?: item-level social influence prediction for users and posts ranking , 2011, SIGIR.

[8]  Tat-Seng Chua,et al.  Social-Sensed Image Search , 2014, TOIS.

[9]  Zhou Su,et al.  What Videos Are Similar with You?: Learning a Common Attributed Representation for Video Recommendation , 2014, ACM Multimedia.

[10]  L. Rothkrantz,et al.  Toward an affect-sensitive multimodal human-computer interaction , 2003, Proc. IEEE.

[11]  Jon M. Kleinberg,et al.  Feedback effects between similarity and social influence in online communities , 2008, KDD.

[12]  Andrew McCallum,et al.  SampleRank: Training Factor Graphs with Atomic Gradients , 2011, ICML.

[13]  Ramesh C. Jain,et al.  Social-Sensed Multimedia Computing , 2016, IEEE Multim..

[14]  Long Jiang,et al.  User-level sentiment analysis incorporating social networks , 2011, KDD.

[15]  Nicu Sebe,et al.  MULTIMODAL EMOTION RECOGNITION , 2005 .

[16]  Takashi Inui,et al.  Extracting Semantic Orientations of Words using Spin Model , 2005, ACL.

[17]  Yun Yang,et al.  User interest and social influence based emotion prediction for individuals , 2013, ACM Multimedia.

[18]  Jiebo Luo,et al.  Cross-modality Consistent Regression for Joint Visual-Textual Sentiment Analysis of Social Multimedia , 2016, WSDM.

[19]  Johan Bollen,et al.  Twitter mood predicts the stock market , 2010, J. Comput. Sci..

[20]  Lillian Lee,et al.  Opinion Mining and Sentiment Analysis , 2008, Found. Trends Inf. Retr..

[21]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[22]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[23]  Allan Hanbury,et al.  Affective image classification using features inspired by psychology and art theory , 2010, ACM Multimedia.

[24]  Brendan T. O'Connor,et al.  From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series , 2010, ICWSM.

[25]  Jie Tang,et al.  Can we understand van gogh's mood?: learning to infer affects from images in social networks , 2012, ACM Multimedia.

[26]  Nicu Sebe,et al.  Emotion Recognition Based on Joint Visual and Audio Cues , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[27]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[28]  Huan Liu,et al.  Exploiting social relations for sentiment analysis in microblogging , 2013, WSDM.

[29]  Ravi Kumar,et al.  Influence and correlation in social networks , 2008, KDD.

[30]  Xing Xie,et al.  User-Service Rating Prediction by Exploring Social Users' Rating Behaviors , 2016, IEEE Transactions on Multimedia.

[31]  Adam Rae,et al.  Prediction of favourite photos using social, visual, and textual signals , 2010, ACM Multimedia.

[32]  L. Ou,et al.  A study of colour emotion and colour preference. Part I: Colour emotions for single colours , 2004 .

[33]  Fangzhao Wu,et al.  Structured microblog sentiment classification via social context regularization , 2016, Neurocomputing.

[34]  Rosalind W. Picard Affective computing: challenges , 2003, Int. J. Hum. Comput. Stud..

[35]  Michael J. Pazzani,et al.  Content-Based Recommendation Systems , 2007, The Adaptive Web.

[36]  Eliot R. Smith,et al.  Situating Social Influence Processes: Dynamic, Multidirectional Flows of Influence Within Social Networks , 2007, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[37]  Nadia Bianchi-Berthouze,et al.  K-DIME: An Affective Image Filtering System , 2003, IEEE Multim..

[38]  Laks V. S. Lakshmanan,et al.  Learning influence probabilities in social networks , 2010, WSDM '10.

[39]  Jiebo Luo,et al.  Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks , 2015, AAAI.

[40]  Erik Cambria,et al.  Fusing audio, visual and textual clues for sentiment analysis from multimodal content , 2016, Neurocomputing.