Hello There! Is Now a Good Time to Talk?

Increasing number of researchers and designers are envisioning a wide range of novel proactive conversational services for smart speakers such as context-aware reminders and restocking household items. When initiating conversational interactions proactively, smart speakers need to consider users' contexts to minimize disruption. In this work, we aim to broaden our understanding of opportune moments for proactive conversational interactions in domestic contexts. Toward this goal, we built a voice-based experience sampling device and conducted a one-week field study with 40 participants living in university dormitories. From 3,572 in-situ user experience reports, we proposed 19 activity categories to investigate contextual factors related to interruptibility. Our data analysis results show that the key determinants for opportune moments are closely related to both personal contextual factors such as busyness, mood, and resource conflicts for dual-tasking, and the other contextual factors associated with the everyday routines at home, including user mobility and social presence. Based on these findings, we discuss the need for designing context-aware proactive conversation management features that dynamically control conversational interactions based on users' contexts and routines.

[1]  Youn-Kyung Lim,et al.  User experience in do-it-yourself-style smart homes , 2015, UbiComp.

[2]  Uichin Lee,et al.  PASS: Reducing Redundant Notifications between a Smartphone and a Smartwatch for Energy Saving , 2020, IEEE Transactions on Mobile Computing.

[3]  Joyce Ho,et al.  Using context-aware computing to reduce the perceived burden of interruptions from mobile devices , 2005, CHI.

[4]  J. Cho,et al.  Reducing Confusion about Grounded Theory and Qualitative Content Analysis: Similarities and Differences , 2014 .

[5]  Rebecca E. Grinter,et al.  How Smart Homes Learn: The Evolution of the Networked Home and Household , 2007, UbiComp.

[6]  LeeUichin,et al.  Multi-Stage Receptivity Model for Mobile Just-In-Time Health Intervention , 2019 .

[7]  Sarah Sharples,et al.  Voice Interfaces in Everyday Life , 2018, CHI.

[8]  Gierad Laput,et al.  Ubicoustics: Plug-and-Play Acoustic Activity Recognition , 2018, UIST.

[9]  Daniel McDuff,et al.  Wearable ESM: differences in the experience sampling method across wearable devices , 2016, MobileHCI.

[10]  E. Hall,et al.  The Hidden Dimension , 1970 .

[11]  Andreas Krause,et al.  SenSay: a context-aware mobile phone , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[12]  John Zimmerman,et al.  Principles of Smart Home Control , 2006, UbiComp.

[13]  Janne Lindqvist,et al.  How Busy Are You?: Predicting the Interruptibility Intensity of Mobile Users , 2017, CHI.

[14]  Uichin Lee,et al.  Interaction Restraint: Enforcing Adaptive Cognitive Tasks to Restrain Problematic User Interaction , 2018, CHI Extended Abstracts.

[15]  Mirco Musolesi,et al.  Designing content-driven intelligent notification mechanisms for mobile applications , 2015, UbiComp.

[16]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.

[17]  Brigitte Meillon,et al.  The Sweet-Home speech and multimodal corpus for home automation interaction , 2014, LREC.

[18]  Jo Vermeulen,et al.  From today's augmented houses to tomorrow's smart homes: new directions for home automation research , 2014, UbiComp.

[19]  Paul Johns,et al.  Focused, Aroused, but so Distractible: Temporal Perspectives on Multitasking and Communications , 2015, CSCW.

[20]  Frank Bentley,et al.  Understanding the Long-Term Use of Smart Speaker Assistants , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[21]  Jaime Teevan,et al.  Supporting Workplace Detachment and Reattachment with Conversational Intelligence , 2018, CHI.

[22]  Mirco Musolesi,et al.  My Phone and Me: Understanding People's Receptivity to Mobile Notifications , 2016, CHI.

[23]  Emre Ertin,et al.  cStress: towards a gold standard for continuous stress assessment in the mobile environment , 2015, UbiComp.

[24]  Wendy Ju,et al.  Is Now A Good Time?: An Empirical Study of Vehicle-Driver Communication Timing , 2019, CHI.

[25]  Uichin Lee,et al.  LocknType: Lockout Task Intervention for Discouraging Smartphone App Use , 2019, CHI.

[26]  William G. Griswold,et al.  Place-Its: A Study of Location-Based Reminders on Mobile Phones , 2005, UbiComp.

[27]  Irene Lopatovska,et al.  Personification of the Amazon Alexa: BFF or a Mindless Companion , 2018, CHIIR.

[28]  LeeUichin,et al.  Interruptibility for In-vehicle Multitasking , 2020 .

[29]  Jason C. Yip,et al.  Communication Breakdowns Between Families and Alexa , 2019, CHI.

[30]  Albrecht Schmidt,et al.  Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[31]  Marco Aurélio Gerosa,et al.  How Should My Chatbot Interact? A Survey on Social Characteristics in Human–Chatbot Interaction Design , 2019, Int. J. Hum. Comput. Interact..

[32]  Min Zhao,et al.  Research on Active Interaction Design for Smart Speakers Agent of Home Service Robot , 2019, HCI.

[33]  Heather Richter Lipford,et al.  I don't own the data": End User Perceptions of Smart Home Device Data Practices and Risks , 2019, SOUPS @ USENIX Security Symposium.

[34]  Paul N. Bennett,et al.  Guidelines for Human-AI Interaction , 2019, CHI.

[35]  Simo Hosio,et al.  Overcoming compliance bias in self-report studies: A cross-study analysis , 2020, Int. J. Hum. Comput. Stud..

[36]  Saul Greenberg,et al.  Proxemic interaction: designing for a proximity and orientation-aware environment , 2010, ITS '10.

[37]  Jennifer Marlow,et al.  Designing for Workplace Reflection: A Chat and Voice-Based Conversational Agent , 2018, Conference on Designing Interactive Systems.

[38]  Uichin Lee,et al.  Multi-Stage Receptivity Model for Mobile Just-In-Time Health Intervention , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[39]  Alain Karsenty,et al.  Unremarkable computing , 2002, CHI.

[40]  Steve Benford,et al.  The evolution of buildings and implications for the design of ubiquitous domestic environments , 2003, CHI '03.

[41]  Anind K. Dey,et al.  Sensors Know When to Interrupt You in the Car: Detecting Driver Interruptibility Through Monitoring of Peripheral Interactions , 2015, CHI.

[42]  Niels Taatgen,et al.  Toward a unified theory of the multitasking continuum: from concurrent performance to task switching, interruption, and resumption , 2009, CHI.

[43]  Francesco Ricci,et al.  A Context-Aware Model for Proactive Recommender Systems in the Tourism Domain , 2015, MobileHCI Adjunct.

[44]  Brian P. Bailey,et al.  Investigating the effectiveness of mental workload as a predictor of opportune moments for interruption , 2005, CHI Extended Abstracts.

[45]  Stuart M. Allen,et al.  Interruptibility prediction for ubiquitous systems: conventions and new directions from a growing field , 2015, UbiComp.

[46]  Daniel Saakes,et al.  The Effects of Interruption Timings on Autonomous Height-Adjustable Desks that Respond to Task Changes , 2019, CHI.

[47]  G. Henri ter Hofte,et al.  Xensible interruptions from your mobile phone , 2007, Mobile HCI.

[48]  Brian P. Bailey,et al.  Effects of intelligent notification management on users and their tasks , 2008, CHI.

[49]  Auk Kim,et al.  Interrupting Drivers for Interactions: Predicting Opportune Moments for In-vehicle Proactive Auditory-verbal Tasks , 2019 .

[50]  Sung-Ju Lee,et al.  Intelligent positive computing with mobile, wearable, and IoT devices: Literature review and research directions , 2019, Ad Hoc Networks.

[51]  A. Mihailidis,et al.  Context-aware assistive devices for older adults with dementia , 2002 .

[52]  Shruti Sannon,et al.  "Alexa is my new BFF": Social Roles, User Satisfaction, and Personification of the Amazon Echo , 2017, CHI Extended Abstracts.

[53]  Nick Feamster,et al.  You, Me, and IoT: How Internet-Connected Home Devices Affect Interpersonal Relationships , 2019, CSCW Companion.

[54]  Uichin Lee,et al.  Cognitive States Matter: Design Guidelines for Driving Situation Awareness in Smart Vehicles , 2020, Sensors.

[55]  Dongman Lee,et al.  Don't Bother Me. I'm Socializing!: A Breakpoint-Based Smartphone Notification System , 2017, CSCW.

[56]  Jodi Forlizzi,et al.  "Hey Alexa, What's Up?": A Mixed-Methods Studies of In-Home Conversational Agent Usage , 2018, Conference on Designing Interactive Systems.

[57]  James Fogarty,et al.  Examining task engagement in sensor-based statistical models of human interruptibility , 2005, CHI.

[58]  Auk Kim,et al.  Interruptibility for In-vehicle Multitasking , 2020, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[59]  Mirco Musolesi,et al.  InterruptMe: designing intelligent prompting mechanisms for pervasive applications , 2014, UbiComp.

[60]  Brian P. Bailey,et al.  If not now, when?: the effects of interruption at different moments within task execution , 2004, CHI.

[61]  T. Kamimura,et al.  Medication Reminder Device for the Elderly Patients With Mild Cognitive Impairment , 2012, American journal of Alzheimer's disease and other dementias.

[62]  Tom Rodden,et al.  Domestic Routines and Design for the Home , 2004, Computer Supported Cooperative Work (CSCW).