Voice as a Contemporary Frontier of Interaction Design

Voice assistants’ increasingly nuanced and natural communication bears new opportunities for user experiences and task automation, while challenging existing patterns of human-computer interaction. A fragmented research field, as well as constant technological advancements, impede a common apprehension of prevalent design features of voice-based interfaces. As part of this study, 86 papers across domains are systematically identified and analysed to arrive at a common understanding of voice assistants. The review highlights perceptual differences to other human-computer interfaces and points out relevant auditory cues. Key findings regarding those cues’ impact on user perception and behaviour are discussed along with the three design strategies 1) personification, 2) individualization and 3) contextualization. Avenues for future research are lastly deducted. Our results provide relevant opportunities to researchers and designers alike to advance the design and deployment of voice assistants.

[1]  Ravi Kuber,et al.  "Siri Talks at You": An Empirical Investigation of Voice-Activated Personal Assistant (VAPA) Usage by Individuals Who Are Blind , 2018, ASSETS.

[2]  Shaun W. Lawson,et al.  Voice as a Design Material: Sociophonetic Inspired Design Strategies in Human-Computer Interaction , 2019, CHI.

[3]  Michael Frankfurter,et al.  A First Look At Communication Theory , 2016 .

[4]  Chinmay Kulkarni,et al.  One Voice Fits All? Social Implications and Research Challenges of Designing Voices for Smart Devices , 2019 .

[5]  L. Lindquist,et al.  Voice‐Controlled Intelligent Personal Assistants to Support Aging in Place , 2019, Journal of the American Geriatrics Society.

[6]  Daniela Mapelli,et al.  Investigating the Accessibility of Voice Assistants With Impaired Users: Mixed Methods Study , 2020, Journal of medical Internet research.

[7]  Andreas Janson,et al.  Value Co-Creation in Smart Services: A Functional Affordances Perspective on Smart Personal Assistants , 2020, J. Assoc. Inf. Syst..

[8]  Helmut Krcmar,et al.  Designing interfaces for multiple-goal environments: Experimental insights from in-vehicle speech interfaces , 2014, TCHI.

[9]  Elfi Furtmueller,et al.  Using grounded theory as a method for rigorously reviewing literature , 2013, Eur. J. Inf. Syst..

[10]  Joonhwan Lee,et al.  It Sounds Like A Woman: Exploring Gender Stereotypes in South Korean Voice Assistants , 2019, CHI Extended Abstracts.

[11]  Wolfgang Minker,et al.  EmpathicSDS: Investigating Lexical and Acoustic Mimicry to Improve Perceived Empathy in Speech Dialogue Systems , 2020, CIU.

[12]  Bilge Mutlu,et al.  Designing persuasive robots: How robots might persuade people using vocal and nonverbal cues , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[13]  Andreas Janson,et al.  Classifying Smart Personal Assistants: An Empirical Cluster Analysis , 2019, HICSS.

[14]  Manfred Tscheligi,et al.  Exploring voice user interfaces for seniors , 2013, PETRA '13.

[15]  Judith A. Hall,et al.  Nonverbal behavior and the vertical dimension of social relations: a meta-analysis. , 2005, Psychological bulletin.

[16]  Andreas Janson,et al.  Developing Lawful Technologies - A Revelatory Case Study on Design Patterns , 2021, HICSS.

[17]  Florian Alt,et al.  At Your Service: Designing Voice Assistant Personalities to Improve Automotive User Interfaces , 2019, CHI.

[18]  J. Leimeister,et al.  Female by Default? – Exploring the Effect of Voice Assistant Gender and Pitch on Trait and Trust Attribution , 2021, CHI Extended Abstracts.

[19]  Alexander Maedche,et al.  A Taxonomy of Social Cues for Conversational Agents , 2019, Int. J. Hum. Comput. Stud..

[20]  Claus-Peter H. Ernst,et al.  The Impact of Gender Stereotyping on the Perceived Likability of Virtual Assistants , 2020, AMCIS.

[21]  Patrícia Augustin Jaques,et al.  The effects of animated pedagogical agents in an English-as-a-foreign-language learning environment , 2016, Int. J. Hum. Comput. Stud..

[22]  Anouk S. Bergner,et al.  Conversational robo advisors as surrogates of trust: onboarding experience, firm perception, and consumer financial decision making , 2020, Journal of the Academy of Marketing Science.

[23]  John D. Bransford,et al.  Gender Representation and Humanoid Robots Designed for Domestic Use , 2009, Int. J. Soc. Robotics.

[24]  Michael F. McTear,et al.  The Rise of the Conversational Interface: A New Kid on the Block? , 2016, FETLT.

[25]  Serge Egelman,et al.  Investigating Users' Preferences and Expectations for Always-Listening Voice Assistants , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[26]  Alan R. Dennis,et al.  Media, Tasks, and Communication Processes: A Theory of Media Synchronicity , 2008, MIS Q..

[27]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[28]  Charles F. Hofacker,et al.  Adding Voice to the Omnichannel and How that Affects Brand Trust , 2019, Journal of Interactive Marketing.

[29]  Taezoon Park,et al.  When stereotypes meet robots: The double-edge sword of robot gender and personality in human-robot interaction , 2014, Comput. Hum. Behav..

[30]  Gil G. Rosenthal,et al.  Visual and acoustic communication in non-human animals: a comparison , 2000, Journal of Biosciences.

[31]  Aaron C. Elkins,et al.  The Sound of Trust: Voice as a Measurement of Trust During Interactions with Embodied Conversational Agents , 2013 .

[32]  Andreas Janson,et al.  Codifying Interdisciplinary Design Knowledge Through Patterns - The Case of Smart Personal Assistants , 2020, DESRIST.

[33]  Gerhard Schwabe,et al.  Studying with the Help of Digital Tutors: Design Aspects of Conversational Agents that Influence the Learning Process , 2020, HICSS.

[34]  P. Belin,et al.  Thinking the voice: neural correlates of voice perception , 2004, Trends in Cognitive Sciences.

[35]  Conor McGinn,et al.  How context shapes the appropriateness of a robot’s voice , 2020, 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN).

[36]  Noah L. Schroeder,et al.  Reconsidering the voice effect when learning from a virtual human , 2017, Comput. Educ..

[37]  Hsi-Peng Lu,et al.  Stereotypes or golden rules? Exploring likable voice traits of social robots as active aging companions for tech-savvy baby boomers in Taiwan , 2018, Comput. Hum. Behav..

[38]  Friederike Eyssel,et al.  ‘If you sound like me, you must be more human’: On the interplay of robot and user features on human-robot acceptance and anthropomorphism , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[39]  Alexander I. Rudnicky,et al.  Towards Improving the Naturalness of Social Conversations with Dialogue Systems , 2010, SIGDIAL Conference.

[40]  F. Niyi Akinnaso,et al.  On The Differences Between Spoken and Written Language , 1982 .

[41]  Xiaojuan Ma,et al.  Designing Emotional Expressions of Conversational States for Voice Assistants: Modality and Engagement , 2018, CHI Extended Abstracts.

[42]  Chidera Obinali.  The Perception of Gender in Voice Assistants , 2019 .

[43]  Alexander Benlian,et al.  Anthropomorphic Information Systems , 2019, Business & Information Systems Engineering.

[44]  C. Nass,et al.  Does computer-synthesized speech manifest personality? Experimental tests of recognition, similarity-attraction, and consistency-attraction. , 2001, Journal of experimental psychology. Applied.

[45]  Selina Jeanne Sutton,et al.  Gender Ambiguous, not Genderless: Designing Gender in Voice User Interfaces (VUIs) with Sensitivity , 2020, CIU.

[46]  Nichola Lubold,et al.  Effects of adapting to user pitch on rapport perception, behavior, and state with a social robotic learning companion , 2020, User Modeling and User-Adapted Interaction.

[47]  Seiji Yamada,et al.  Non-humanlike Spoken Dialogue: A Design Perspective , 2010, SIGDIAL Conference.

[48]  Ilaria Torre,et al.  Trust in artificial voices: A "congruency effect" of first impressions and behavioural experience , 2018, APAScience.

[49]  Gabriele Trovato,et al.  Security and guidance: Two roles for a humanoid robot in an interaction experiment , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[50]  Bruce A. MacDonald,et al.  The Effects of Synthesized Voice Accents on User Perceptions of Robots , 2011, Int. J. Soc. Robotics.

[51]  D. Rubin,et al.  Reading and listening to oral‐based versus literate‐based discourse , 2000 .

[52]  Benjamin R. Cowan,et al.  Revolution or Evolution? Speech Interaction and HCI Design Guidelines , 2019, IEEE Pervasive Computing.

[53]  Roland T. Rust,et al.  Engaged to a Robot? The Role of AI in Service , 2020, Journal of Service Research.

[54]  Ilaria Torre,et al.  If your device could smile: People trust happy-sounding artificial agents more , 2020, Comput. Hum. Behav..

[55]  Richard T. Watson,et al.  Analyzing the Past to Prepare for the Future: Writing a Literature Review , 2002, MIS Q..

[56]  Anja Gampe,et al.  Communicative and social consequences of interactions with voice assistants , 2020, Comput. Hum. Behav..

[57]  Robert K. Atkinson,et al.  Animated pedagogical agents: does their degree of embodiment impact learning from static or animated worked examples? , 2007 .

[58]  Margot Brereton,et al.  Use of voice activated interfaces by people with intellectual disability , 2018, OZCHI.

[59]  Izak Benbasat,et al.  Online Consumer Trust and Live Help Interfaces: The Effects of Text-to-Speech Voice and Three-Dimensional Avatars , 2005, Int. J. Hum. Comput. Interact..

[60]  Joost Broekens,et al.  Towards Understanding the Effect of Voice on Human-Agent Negotiation , 2020, IVA.

[61]  Izak Benbasat,et al.  Understanding Customer Trust in Agent-Mediated Electronic Commerce, Web-Mediated Electronic Commerce, and Traditional Commerce , 2004, Inf. Technol. Manag..

[62]  Eugene Cho,et al.  The Effects of Modality, Device, and Task Differences on Perceived Human Likeness of Voice-Activated Virtual Assistants , 2019, Cyberpsychology Behav. Soc. Netw..

[63]  Duncan P. Brumby,et al.  "Watch Out!": Semi-Autonomous Vehicles Using Assertive Voices to Grab Distracted Drivers' Attention , 2019, CHI Extended Abstracts.

[64]  S. Kopp,et al.  Age-Related Differences in the Evaluation of a Virtual Health Agent’s Appearance and Embodiment in a Health-Related Interaction: Experimental Lab Study , 2019, Journal of medical Internet research.

[65]  Ryan M. Schuetzler,et al.  Deciding Whether and How to Deploy Chatbots , 2021, MIS Q. Executive.

[66]  Fiorella de Rosis,et al.  Evaluating a realistic agent in an advice-giving task , 2005, Int. J. Hum. Comput. Stud..

[67]  Benjamin R. Cowan,et al.  Towards Understanding How Speech Output Affects Navigation System Credibility , 2016, CHI Extended Abstracts.

[68]  Daniel Buschek,et al.  Developing a Personality Model for Speech-based Conversational Agents Using the Psycholexical Approach , 2020, CHI.

[69]  Stacy M. Branham,et al.  Reading Between the Guidelines: How Commercial Voice Assistant Guidelines Hinder Accessibility for Blind Users , 2019, ASSETS.

[70]  Julia Hirschberg,et al.  Turn-taking cues in task-oriented dialogue , 2011, Comput. Speech Lang..

[71]  Victor R. Prybutok,et al.  An Integrated Model of Voice-User Interface Continuance Intention: The Gender Effect , 2018, Int. J. Hum. Comput. Interact..