Engagement-aware computing: modelling user engagement from mobile contexts

In this paper, we examine the potential of using mobile context to model user engagement. Taking an experimental approach, we systematically explore the dynamics of user engagement with a smartphone through three different studies. Specifically, to understand the feasibility of detecting user engagement from mobile context, we first assess an EEG artifact with 10 users and observe a strong correlation between automatically detected engagement scores and user's subjective perception of engagement. Grounded on this result, we model a set of application level features derived from smartphone usage of 10 users to detect engagement of a usage session using a Random Forest classifier. Finally, we apply this model to train a variety of contextual factors acquired from smartphone usage logs of 130 users to predict user engagement using an SVM classifier with a F1-Score of 0.82. Our experimental results highlight the potential of mobile contexts in designing engagement-aware applications and provide guidance to future explorations.

[1]  Christopher G. Atkeson,et al.  Predicting human interruptibility with sensors , 2005, TCHI.

[2]  William Stephenson,et al.  The Play Theory of Mass Communication , 1967 .

[3]  Tadashi Okoshi,et al.  Attelia: Reducing user's cognitive load due to interruptive notifications on smart phones , 2015, 2015 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[4]  Christopher A. Monk,et al.  The Attentional Costs of Interrupting Task Performance at Various Stages , 2002 .

[5]  Monroe C. Beardsley,et al.  The Aesthetic Point of View: Selected Essays , 1984 .

[6]  Alastair R. Beresford,et al.  Device Analyzer: Understanding Smartphone Usage , 2013, MobiQuitous.

[7]  M. Csíkszentmihályi.  Toward a Psychology of Optimal Experience , 2014 .

[8]  Steve Benford,et al.  Investigating episodes of mobile phone activity as indicators of opportune moments to deliver notifications , 2011, Mobile HCI.

[9]  Ouri Wolfson,et al.  Extracting Semantic Location from Outdoor Positioning Systems , 2006, 7th International Conference on Mobile Data Management (MDM'06).

[10]  Michael S. Bernstein,et al.  Pay It Backward: Per-Task Payments on Crowdsourcing Platforms Reduce Productivity , 2016, CHI.

[11]  Brian P. Bailey,et al.  Effects of intelligent notification management on users and their tasks , 2008, CHI.

[12]  Paul Johns,et al.  Bored mondays and focused afternoons: the rhythm of attention and online activity in the workplace , 2014, CHI.

[13]  Paul A. Cairns,et al.  An empirical evaluation of the User Engagement Scale (UES) in online news environments , 2015, Inf. Process. Manag..

[14]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Yunxin Liu,et al.  MoodScope: building a mood sensor from smartphone usage patterns , 2013, MobiSys.

[16]  Anind K. Dey,et al.  ProactiveTasks: the short of mobile device use sessions , 2014, MobileHCI '14.

[17]  Steve Benford,et al.  Effects of content and time of delivery on receptivity to mobile interruptions , 2010, Mobile HCI.

[18]  Peter Chapman,et al.  Engagement in multimedia training systems , 1999, Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers.

[19]  Jason Flinn,et al.  The Case for Operating System Management of User Attention , 2015, HotMobile.

[20]  Alex Pentland,et al.  Daily Stress Recognition from Mobile Phone Data, Weather Conditions and Individual Traits , 2014, ACM Multimedia.

[21]  Heather L. O'Brien,et al.  The influence of hedonic and utilitarian motivations on user engagement: The case of online shopping experiences , 2010, Interact. Comput..

[22]  Xiaoli Wang,et al.  Mobility tracking using GPS, Wi-Fi and Cell ID , 2012, The International Conference on Information Network 2012.

[23]  Yvonne Rogers,et al.  Mood Squeezer: Lightening up the Workplace through Playful and Lightweight Interactions , 2015, CSCW.

[24]  Elaine Toms,et al.  The development and evaluation of a survey to measure user engagement , 2010, J. Assoc. Inf. Sci. Technol..

[25]  Elaine Toms,et al.  Information interaction: Providing a framework for information architecture , 2002, J. Assoc. Inf. Sci. Technol..

[26]  J. Kruschke.  Toward a unified model of attention in associative learning , 2001 .

[27]  Johan F. Hoorn,et al.  Realism is not all! User engagement with task-related interface characters , 2007, Interact. Comput..

[28]  Mirco Musolesi,et al.  InterruptMe: designing intelligent prompting mechanisms for pervasive applications , 2014, UbiComp.

[29]  Yiannis Gkoufas,et al.  Predicting User Engagement in Twitter with Collaborative Ranking , 2014, RecSysChallenge '14.

[30]  Ehsan Tarkesh Esfahani,et al.  Using Brain-Computer Interfaces to Detect Human Satisfaction in Human-Robot Interaction , 2011, Int. J. Humanoid Robotics.

[31]  Jorge Gonçalves,et al.  Revisitation analysis of smartphone app use , 2015, UbiComp.

[32]  Ricardo Baeza-Yates,et al.  Measuring inter-site engagement , 2013, 2013 IEEE International Conference on Big Data.

[33]  Eric N. Wiebe,et al.  Measuring engagement in video game-based environments: Investigation of the User Engagement Scale , 2014, Comput. Hum. Behav..

[34]  Jorge Gonçalves,et al.  Contextual experience sampling of mobile application micro-usage , 2014, MobileHCI '14.

[35]  M. Csíkszentmihályi.  Flow and the Foundations of Positive Psychology , 2014 .

[36]  Brian P. Bailey,et al.  Oasis: A framework for linking notification delivery to the perceptual structure of goal-directed tasks , 2010, TCHI.

[37]  Duncan J. Watts,et al.  Financial incentives and the "performance of crowds" , 2009, HCOMP '09.

[38]  Jorge Gonçalves,et al.  A Systematic Assessment of Smartphone Usage Gaps , 2016, CHI.

[39]  Elaine Toms,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008, J. Assoc. Inf. Sci. Technol..

[40]  Brian P. Bailey,et al.  On the need for attention-aware systems: Measuring effects of interruption on task performance, error rate, and affective state , 2006, Comput. Hum. Behav..

[41]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[42]  Sidney K. D'Mello,et al.  Detecting boredom and engagement during writing with keystroke analysis, task appraisals, and stable traits , 2013, IUI '13.

[43]  Dahui Li,et al.  Task Design, Motivation, and Participation in Crowdsourcing Contests , 2011, Int. J. Electron. Commer..

[44]  Rosalind W. Picard.  Affective Computing: From Laughter to IEEE , 2010 .

[45]  Fabio Pianesi,et al.  Happiness Recognition from Mobile Phone Data , 2013, 2013 International Conference on Social Computing.

[46]  Brian P. Bailey,et al.  The Effects of Interruptions on Task Performance, Annoyance, and Anxiety in the User Interface , 2001, INTERACT.

[47]  Ricardo Baeza-Yates,et al.  Predicting The Next App That You Are Going To Use , 2015, WSDM.

[48]  Yelena Tsarenko,et al.  Understanding Consumer Intention to Use Mobile Services , 2010 .

[49]  Martin Pielot,et al.  When attention is not scarce - detecting boredom from mobile phone usage , 2015, UbiComp.

[50]  Mirco Musolesi,et al.  My Phone and Me: Understanding People's Receptivity to Mobile Notifications , 2016, CHI.