MoodExplorer

Social psychology and neuroscience had confirmed that emotion state exerts a significant effect on human communication, perception, social behavior and decision making. With the wide availability of smartphones equipped with microphone, accelerometer, GPS, and other source of sensors, it is worthwhile to explore the possibility of automatic emotion detection via smartphone sensing. Particularly, we focus on a novel research problem that tries to detect the compound emotion (a set of multiple dimensional basic emotions) of smartphone users. We observe that users' self-reported emotional states have high correlation with their smartphone usage patterns and sensing data. Based on the observations, we exploit a feature extraction and selection algorithm to find the most significant features. We further adopt a factor graph model to tackle the correlations between features and emotion labels, and propose a machine learning algorithm for compound emotion detection based on the smartphone sensing data. The proposed mechanism is implemented as an APP called MoodExplorer in Android platform. Extensive experiments conducted on the smartphone data collected from 30 university students show that MoodExplorer can recognize users' compound emotions with 76.0% exact match on average.

[1]  Kasper Hornbæk,et al.  An affect detection technique using mobile commodity sensors in the wild , 2016, UbiComp.

[2]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[3]  Constantinos Patsakis,et al.  A survey on mobile affective computing , 2017, Comput. Sci. Rev..

[4]  Cecilia Mascolo,et al.  Mobile Sensing at the Service of Mental Well-being: a Large-scale Longitudinal Study , 2017, WWW.

[5]  Romit Roy Choudhury,et al.  Your reactions suggest you liked the movie: automatic content rating via reaction sensing , 2013, UbiComp.

[6]  Thomas Stütz,et al.  Smartphone Based Stress Prediction , 2015, UMAP.

[7]  Lei Zheng,et al.  DeepMood: Modeling Mobile Phone Typing Dynamics for Mood Detection , 2017, KDD.

[8]  Sinan Aral,et al.  Identifying Influential and Susceptible Members of Social Networks , 2012, Science.

[9]  M. Levandowsky,et al.  Distance between Sets , 1971, Nature.

[10]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[11]  Larry Wasserman,et al.  All of Statistics: A Concise Course in Statistical Inference , 2004 .

[12]  Michael Beigl,et al.  Smartphone-Based Detection of Location Changes Using WiFi Data , 2016, MobiHealth.

[13]  Alex Pentland,et al.  Daily Stress Recognition from Mobile Phone Data, Weather Conditions and Individual Traits , 2014, ACM Multimedia.

[14]  Andreas Stolcke,et al.  Prosody-based automatic detection of annoyance and frustration in human-computer dialog , 2002, INTERSPEECH.

[15]  Y. Lin,et al.  An Experimental Study on Physiological Parameters Toward Driver Emotion Recognition , 2007, HCI.

[16]  O. John,et al.  Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. , 2003, Journal of personality and social psychology.

[17]  Kiraz Candan Herdem.  Reactions: Twitter based mobile application for awareness of friends' emotions , 2012, UbiComp.

[18]  Tom Geller.  How do you feel?: your computer knows , 2014, CACM.

[19]  Yoshihiko Suhara,et al.  DeepMood: Forecasting Depressed Mood Based on Self-Reported Histories via Recurrent Neural Networks , 2017, WWW.

[20]  P. Ekman,et al.  DIFFERENCES Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion , 2004 .

[21]  Tsuhan Chen,et al.  The painful face - Pain expression recognition using active appearance models , 2009, Image Vis. Comput..

[22]  L. Zhu,et al.  Towards Mood Based Mobile Services and Applications , 2007, EuroSSC.

[23]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[24]  Andrew McCallum,et al.  An Introduction to Conditional Random Fields , 2010, Found. Trends Mach. Learn..

[25]  G. Horstmann,et al.  Coherence between Emotion and Facial Expression: Evidence from Laboratory Experiments , 2013 .

[26]  J. Henry,et al.  The positive and negative affect schedule (PANAS): construct validity, measurement properties and normative data in a large non-clinical sample. , 2004, The British journal of clinical psychology.

[27]  P. Terry,et al.  Distinctions between emotion and mood , 2005 .

[28]  Xin Geng,et al.  Emotion Distribution Recognition from Facial Expressions , 2015, ACM Multimedia.

[29]  Yong Tao,et al.  Compound facial expressions of emotion , 2014, Proceedings of the National Academy of Sciences.

[30]  Lei Huang,et al.  Sentence-level Emotion Classification with Label and Context Dependence , 2015, ACL.

[31]  N. Christakis,et al.  Dynamic spread of happiness in a large social network: longitudinal analysis over 20 years in the Framingham Heart Study , 2008, BMJ : British Medical Journal.

[32]  Richard E. Lucas,et al.  Personality, culture, and subjective well-being: emotional and cognitive evaluations of life. , 2003, Annual review of psychology.

[33]  S. Mourato,et al.  Happiness is greater in natural environments , 2013 .

[34]  Yiran Chen,et al.  MoodCast: Emotion Prediction via Dynamic Continuous Factor Graph Model , 2010, 2010 IEEE International Conference on Data Mining.

[35]  Newton Spolaôr,et al.  ReliefF for Multi-label Feature Selection , 2013, 2013 Brazilian Conference on Intelligent Systems.

[36]  Mirco Musolesi,et al.  Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis , 2015, UbiComp.

[37]  Jennifer Golbeck,et al.  Predicting personality with social media , 2011, CHI Extended Abstracts.

[38]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[39]  Yunxin Liu,et al.  MoodScope: building a mood sensor from smartphone usage patterns , 2013, MobiSys.

[40]  Brendan J. Frey,et al.  A Revolution: Belief Propagation in Graphs with Cycles , 1997, NIPS.

[41]  Wenzhong Li,et al.  Cooperative Positioning and Tracking in Disruption Tolerant Networks , 2015, IEEE Transactions on Parallel and Distributed Systems.

[42]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  S. Stansfeld,et al.  Noise pollution: non-auditory effects on health. , 2003, British medical bulletin.

[44]  R. Plutchik.  A GENERAL PSYCHOEVOLUTIONARY THEORY OF EMOTION , 1980 .

[45]  J. Russell,et al.  The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology , 2005, Development and Psychopathology.

[46]  Grigorios Tsoumakas,et al.  Mining Multi-label Data , 2010, Data Mining and Knowledge Discovery Handbook.

[47]  Newton Spolaôr,et al.  A Comparison of Multi-label Feature Selection Methods using the Problem Transformation Approach , 2013, CLEI Selected Papers.

[48]  S. Tomkins.  Illuminating and Stimulating. (Book Reviews: Affect, Imagery, Consciousness. vol. 1, The Positive Affects) , 1963 .

[49]  Hongxun Yao,et al.  Predicting Continuous Probability Distribution of Image Emotions in Valence-Arousal Space , 2015, ACM Multimedia.

[50]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[51]  Javier Hernandez,et al.  Mood meter: counting smiles in the wild , 2012, UbiComp.

[52]  Yunhao Liu,et al.  iSelf: Towards cold-start emotion labeling using transfer learning with smartphones , 2015, INFOCOM.

[53]  Jie Tang,et al.  Social Role-Aware Emotion Contagion in Image Social Networks , 2016, AAAI.

[54]  Fadel Adib,et al.  Emotion recognition using wireless signals , 2016, MobiCom.

[55]  J. M. Hammersley,et al.  Markov fields on finite graphs and lattices , 1971 .