Studying Breakdowns in Interactions with Smart Speakers

The popularity of voice-controlled smart speakers with intelligent personal assistants (IPAs) like the Amazon Echo and their increasing use as an interface for other Internet of Things (IoT) technologies in the home provides opportunities to study smart speakers as an emerging and ubiquitous IoT device. Prior research has studied how smart speaker usage has unfolded in homes and how the devices have been integrated into people's daily routines. In this context, researchers have also observed instances of smart speakers' 'black box' behaviour. In this position paper, we present findings from a study we conducted to specifically investigate such challenges people experience around intelligibility and control of their smart speakers, for instance, when the smart speaker interfaces with other IoT systems. Reflecting on our findings, we discuss new possible directions for smart speakers including physical intelligibility, situational physical interaction, and providing access to alternative interpretations in shared and non-shared contexts.

[1]  Sarah Sharples,et al.  "Do Animals Have Accents?": Talking with Agents in Multi-Party Conversation , 2017, CSCW.

[2]  Miguel Bruns Alonso,et al.  Ripple Thermostat: Affecting the Emotional Experience through Interactive Force Feedback and Shape Change , 2018, CHI.

[3]  Karin Coninx,et al.  PervasiveCrystal: Asking and Answering Why and Why Not Questions about Pervasive Computing Applications , 2010, 2010 Sixth International Conference on Intelligent Environments.

[4]  Anind K. Dey,et al.  Why and why not explanations improve the intelligibility of context-aware intelligent systems , 2009, CHI.

[5]  Sarah Sharples,et al.  Voice Interfaces in Everyday Life , 2018, CHI.

[6]  Sarvapali D. Ramchurn,et al.  The potential of physical motion cues: changing people's perception of robots' performance , 2016, UbiComp.

[7]  Anind K. Dey,et al.  Design of an intelligible mobile context-aware application , 2011, Mobile HCI.

[8]  Keith Cheverst,et al.  Exploring Issues of User Model Transparency and Proactive Behaviour in an Office Environment Control System , 2005, User Modeling and User-Adapted Interaction.

[9]  W. Keith Edwards,et al.  Intelligibility and Accountability: Human Considerations in Context-Aware Systems , 2001, Hum. Comput. Interact..

[10]  Sarvapali D. Ramchurn,et al.  Seeing (Movement) is Believing: The Effect of Motion on Perception of Automatic Systems Performance , 2021, Hum. Comput. Interact..

[11]  Wendy Ju.  Approachability : How People Interpret Automatic Door Movement as Gesture , 2008 .

[12]  Anind K. Dey,et al.  Support for context-aware intelligibility and control , 2009, CHI.

[13]  Josephine Lau,et al.  Alexa, Are You Listening? , 2018, Proc. ACM Hum. Comput. Interact..

[14]  Anind K. Dey,et al.  Toolkit to support intelligibility in context-aware applications , 2010, UbiComp.

[15]  Sarvapali D. Ramchurn,et al.  Tariff Agent , 2016, ACM Trans. Comput. Hum. Interact..

[16]  Austin Henderson,et al.  Making sense of sensing systems: five questions for designers and researchers , 2002, CHI.

[17]  Kris Luyten,et al.  I Bet You Look Good on the Wall: Making the Invisible Computer Visible , 2009, AmI.

[18]  Mark W. Newman,et al.  Learning from a learning thermostat: lessons for intelligent systems for the home , 2013, UbiComp.