Opportunistic and Context-Aware Affect Sensing on Smartphones

The authors identify recent advances, key solutions for implementing opportunistic sensing on smart phones. They explore robustness issues and the challenges of mental health patients as participants.

[1]  Juan Miguel García-Gómez,et al.  A Mobile Health Application to Predict Postpartum Depression Based on Machine Learning. , 2015, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[2]  Agata Kolakowska,et al.  A review of emotion recognition methods based on keystroke dynamics and mouse movements , 2013, 2013 6th International Conference on Human System Interactions (HSI).

[3]  Alex Pentland,et al.  VibeFones: Socially Aware Mobile Phones , 2006, 2006 10th IEEE International Symposium on Wearable Computers.

[4]  Kwang-Ting Cheng,et al.  Using mobile GPU for general-purpose computing – a case study of face recognition on smartphones , 2011, Proceedings of 2011 International Symposium on VLSI Design, Automation and Test.

[5]  Latanya Sweeney,et al.  k-Anonymity: A Model for Protecting Privacy , 2002, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[6]  T. Eerola,et al.  A comparison of the discrete and dimensional models of emotion in music , 2011 .

[7]  Florian Schaub,et al.  Territorial privacy in ubiquitous computing , 2011, 2011 Eighth International Conference on Wireless On-Demand Network Systems and Services.

[8]  Wen Hu,et al.  Poster abstract: Distributed sparse approximation for frog sound classification , 2012, 2012 ACM/IEEE 11th International Conference on Information Processing in Sensor Networks (IPSN).

[9]  B. Prabhakaran,et al.  Real-Time Mobile Facial Expression Recognition System -- A Case Study , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[10]  George C. Patton,et al.  A mobile phone program to track young people’s experiences of mood, stress and coping , 2009, Social Psychiatry and Psychiatric Epidemiology.

[11]  Michel C. A. Klein,et al.  An Ambient Intelligent Agent for Relapse and Recurrence Monitoring in Unipolar Depression , 2009, AIME.

[12]  Klára Vicsi,et al.  Automatic Classification of Emotions in Spontaneous Speech , 2010, COST 2102 Conference.

[13]  Ulrich Reimer,et al.  Smart Coping with Stress: Biofeedback via Smart Phone for Stress Reduction and Relapse Prevention in Alcohol Dependent Subjects , 2011, Bled eConference.

[14]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Qing Guo,et al.  Balancing energy, latency and accuracy for mobile sensor data classification , 2011, SenSys.

[16]  Andrea Gaggioli,et al.  A mobile data collection platform for mental health research , 2013, Personal and Ubiquitous Computing.

[17]  Tomáš Novák,et al.  ITAREPS: Information Technology Aided Relapse Prevention Programme in Schizophrenia , 2008, Schizophrenia Research.

[18]  Jie Liu,et al.  SpeakerSense: Energy Efficient Unobtrusive Speaker Identification on Mobile Phones , 2011, Pervasive.

[19]  Wanda Pratt,et al.  Healthcare in the pocket: Mapping the space of mobile-phone health interventions , 2012, J. Biomed. Informatics.

[20]  P. Ekman,et al.  What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) , 2005 .

[21]  Pawel Prociow,et al.  Mobile psychiatry: towards improving the care for bipolar disorder , 2012, International Journal of Mental Health Systems.

[22]  Jeffrey F. Cohn,et al.  The Timing of Facial Motion in posed and Spontaneous Smiles , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[23]  Andrea Gaggioli,et al.  From Mobile Mental Health to Mobile Wellbeing: Opportunities and Challenges , 2013, MMVR.

[24]  Joseph Kee-Yin Ng,et al.  SmartMood: Toward Pervasive Mood Tracking and Analysis for Manic Episode Detection , 2015, IEEE Transactions on Human-Machine Systems.

[25]  Fanglin Chen,et al.  StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones , 2014, UbiComp.

[26]  Gwen Littlewort,et al.  Recognizing facial expression: machine learning and application to spontaneous behavior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Johan Scholten,et al.  Opportunistic Sensing in Wireless Sensor Networks , 2011, ICON 2011.

[29]  Cecilia Mascolo,et al.  DSP.Ear: leveraging co-processor support for continuous audio sensing on smartphones , 2014, SenSys.

[30]  J. Cohn,et al.  Impact of depression on response to comedy: a dynamic facial coding analysis. , 2007, Journal of abnormal psychology.

[31]  Moushumi Sharmin,et al.  Estimating Drivers' Stress from GPS Traces , 2014, AutomotiveUI.

[32]  R. McCrae,et al.  An introduction to the five-factor model and its applications. , 1992, Journal of personality.

[33]  Mi Zhang,et al.  BodyBeat: a mobile system for sensing non-speech body sounds , 2014, MobiSys.

[34]  Hosub Lee,et al.  Towards unobtrusive emotion recognition for affective social communication , 2012, 2012 IEEE Consumer Communications and Networking Conference (CCNC).

[35]  Emiliano Miluzzo,et al.  People-centric urban sensing , 2006, WICON '06.

[36]  Ellen Thackery,et al.  The Gale encyclopedia of mental disorders , 2002 .

[37]  Peter Leijdekkers,et al.  Design of emotion-aware mobile apps for autistic children , 2014 .

[38]  Sridha Sridharan,et al.  Dynamic texture reconstruction from sparse codes for unusual event detection in crowded scenes , 2011, J-MRE '11.

[39]  D. Mohr,et al.  Harnessing Context Sensing to Develop a Mobile Intervention for Depression , 2011, Journal of medical Internet research.

[40]  Wen Hu,et al.  Nonuniform Compressive Sensing for Heterogeneous Wireless Sensor Networks , 2013, IEEE Sensors Journal.

[41]  C. Darwin,et al.  The Expression of the Emotions in Man and Animals , 1872 .

[42]  Sridha Sridharan,et al.  Sparse Temporal Representations for Facial Expression Recognition , 2011, PSIVT.

[43]  Yunxin Liu,et al.  MoodScope: building a mood sensor from smartphone usage patterns , 2013, MobiSys.

[44]  John Zimmerman,et al.  Detection of Behavior Change in People with Depression , 2014, AAAI Workshop: Modern Artificial Intelligence for Health Analytics.

[45]  Raisa B Gul,et al.  Clinical trials: the challenge of recruitment and retention of participants. , 2010, Journal of clinical nursing.

[46]  Rana El Kaliouby,et al.  When Human Coders (and Machines) Disagree on the Meaning of Facial Affect in Spontaneous Videos , 2009, IVA.

[47]  Yung-Hsiang Lu,et al.  Cloud Computing for Mobile Users: Can Offloading Computation Save Energy? , 2010, Computer.

[48]  Salil S. Kanhere,et al.  A survey on privacy in mobile participatory sensing applications , 2011, J. Syst. Softw..

[49]  Andrew T. Campbell,et al.  Bewell: A smartphone application to monitor, model and promote wellbeing , 2011, PervasiveHealth 2011.

[50]  L. Tennakoon,et al.  Challenges in recruitment of research participants , 2003 .

[51]  Paul Lukowicz,et al.  Can smartphones detect stress-related changes in the behaviour of individuals? , 2012, 2012 IEEE International Conference on Pervasive Computing and Communications Workshops.

[52]  Emiliano Miluzzo,et al.  A survey of mobile phone sensing , 2010, IEEE Communications Magazine.

[53]  Fabio Pianesi,et al.  Happiness Recognition from Mobile Phone Data , 2013, 2013 International Conference on Social Computing.

[54]  García-GómezJuan Miguel,et al.  A Mobile Health Application to Predict Postpartum Depression Based on Machine Learning , 2015 .

[55]  P. Ekman,et al.  DIFFERENCES Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion , 2004 .

[56]  J. Russell.  A circumplex model of affect. , 1980 .

[57]  Oscar Mayora-Ibarra,et al.  Smartphone-Based Recognition of States and State Changes in Bipolar Disorder Patients , 2015, IEEE Journal of Biomedical and Health Informatics.

[58]  Michael Wagner,et al.  Multimodal assistive technologies for depression diagnosis and monitoring , 2013, Journal on Multimodal User Interfaces.

[59]  A. Mehrabian.  Communication without words , 1968 .

[60]  Roland Göcke,et al.  An Investigation of Depressed Speech Detection: Features and Normalization , 2011, INTERSPEECH.

[61]  Véronique Aubergé,et al.  Exploring the graded structure of vocal emotion expressions , 2009 .

[62]  Timothy F. Cootes,et al.  Real-Time Facial Feature Tracking on a Mobile Device , 2011, International Journal of Computer Vision.

[63]  David Wiljer,et al.  Youth Mental Health Interventions via Mobile Phones: A Scoping Review , 2014, Cyberpsychology Behav. Soc. Netw..

[64]  Daniel Gatica-Perez,et al.  StressSense: detecting stress in unconstrained acoustic environments using smartphones , 2012, UbiComp.

[65]  N. Ambady,et al.  Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. , 1992 .

[66]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Roland Göcke,et al.  An approach for automatically measuring facial activity in depressed subjects , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[68]  Igor Bisio,et al.  Gender-Driven Emotion Recognition Through Speech Signals For Ambient Intelligence Applications , 2013, IEEE Transactions on Emerging Topics in Computing.

[69]  Christoph Busch,et al.  Unobtrusive User-Authentication on Mobile Phones Using Biometric Gait Recognition , 2010, 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[70]  P. Ekman.  Universals and cultural differences in facial expressions of emotion. , 1972 .

[71]  Meng-Chieh Chiu,et al.  Leveraging graphical models to improve accuracy and reduce privacy risks of mobile sensing , 2013, MobiSys '13.

[72]  Akane Sano,et al.  Stress Recognition Using Wearable Sensors and Mobile Phones , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[73]  Shaogang Gong,et al.  Robust facial expression recognition using local binary patterns , 2005, IEEE International Conference on Image Processing 2005.

[74]  P. Ekman,et al.  Emotion in the Human Face: Guidelines for Research and an Integration of Findings , 1972 .

[75]  Ulrich Reimer,et al.  SmartCoping - A Mobile Solution for Stress Recognition and Prevention , 2014, HEALTHINF.

[76]  Wen Hu,et al.  An Adaptive Algorithm for Compressive Approximation of Trajectory (AACAT) for Delay Tolerant Networks , 2011, EWSN.

[77]  P. Lukowicz,et al.  Towards smart phone based monitoring of bipolar disorder , 2012, mHealthSys '12.

[78]  Maja Pantic,et al.  Fully Automatic Recognition of the Temporal Phases of Facial Actions , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[79]  Lucy Yardley,et al.  Opportunities and Challenges for Smartphone Applications in Supporting Health Behavior Change: Qualitative Study , 2013, Journal of medical Internet research.

[80]  Daniel Gatica-Perez,et al.  Who's Who with Big-Five: Analyzing and Classifying Personality Traits with Smartphones , 2011, 2011 15th Annual International Symposium on Wearable Computers.

[81]  Wen Hu,et al.  Distributed sparse approximation for frog sound classification , 2012, IPSN.

[82]  Shrikanth S. Narayanan,et al.  Toward detecting emotions in spoken dialogs , 2005, IEEE Transactions on Speech and Audio Processing.

[83]  A. Muaremi,et al.  Towards Measuring Stress with Smartphones and Wearable Devices During Workday and Sleep , 2013, BioNanoScience.

[84]  Kazumasa Takami,et al.  A Method of Inferring the Preferences and Mood of Mobile Phone Users by Analyzing Pictograms and Emoticons Used in their Emails , 2010, 2010 Third International Conference on Advances in Human-Oriented and Personalized Mechanisms, Technologies and Services.

[85]  Heng Wang,et al.  Depression recognition based on dynamic facial and vocal expression features using partial least square regression , 2013, AVEC@ACM Multimedia.

[86]  Martin Guha.  The Gale Encyclopedia of Mental Disorders , 2003 .

[87]  Wen Hu,et al.  Real-time classification via sparse representation in acoustic sensor networks , 2013, SenSys '13.

[88]  J. Canny,et al.  AMMON : A Speech Analysis Library for Analyzing Affect , Stress , and Mental Health on Mobile Phones , 2011 .

[89]  Lars Kai Hansen,et al.  Smartphones Get Emotional: Mind Reading Images and Reconstructing the Neural Sources , 2011, ACII.

[90]  K. Scherer.  Vocal affect expression: a review and a model for future research. , 1986, Psychological bulletin.

[91]  Rosalind W. Picard.  Affective Computing for HCI , 1999, HCI.

[92]  Tanzeem Choudhury,et al.  Personal Informatics Can Be Stressful: Collecting, Reflecting, and Embedding Stress Data in Personal Informatics , 2013 .

[93]  In-Ho Choi,et al.  Robust Facial Expression Recognition Using a Smartphone Working against Illumination Variation , 2012 .

[94]  Cecilia Mascolo,et al.  EmotionSense: a mobile phones based adaptive platform for experimental social psychology research , 2010, UbiComp.

[95]  V. Alexandratos,et al.  Mobile Real-Time Stress Detection , 2014 .

[96]  Yuan Gao,et al.  What Does Touch Tell Us about Emotions in Touchscreen-Based Gameplay? , 2012, TCHI.

[97]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.