Listen Only When Spoken To: Interpersonal Communication Cues as Smart Speaker Privacy Controls

Abstract Internet of Things and smart home technologies pose challenges for providing effective privacy controls to users, as smart devices lack both traditional screens and input interfaces. We investigate the potential for leveraging interpersonal communication cues as privacy controls in the IoT context, in particular for smart speakers. We propose privacy controls based on two kinds of interpersonal communication cues – gaze direction and voice volume level – that only selectively activate a smart speaker’s microphone or voice recognition when the device is being addressed, in order to avoid constant listening and speech recognition by the smart speaker microphones and reduce false device activation. We implement these privacy controls in a smart speaker prototype and assess their feasibility, usability and user perception in two lab studies. We find that privacy controls based on interpersonal communication cues are practical, do not impair the smart speaker’s functionality, and can be easily used by users to selectively mute the microphone. Based on our findings, we discuss insights regarding the use of interpersonal cues as privacy controls for smart speakers and other IoT devices.