Explaining automated environments: interrogating scripts, logs, and provenance using voice-assistants

The grand vision of pervasive computing and the Internet of Things (IoT) involves providing people with a range of seamless functionality, be it through automation, information delivery, etc. However, much of the IoT is opaque; it is often difficult for users to uncover and understand how and why particular functionality occurs, the sources of information, the entities involved, and so forth. We argue that automation scripts, as well as logs and provenance records could be leveraged to assist in illuminating the workings of connected and automated environments. This paper explores the use of voice assistants (an accessible, intuitive and increasingly common interface) as a means for allowing users to interrogate what is happening in the IoT systems that surround them. In presenting an exploratory Alexa 'Skill', we discuss several considerations for the implementation of such a system. This work represents a starting point for considering how such assistants could help people better understand---and indeed, evaluate, challenge, and accept---technology that is increasingly pervading our world.

[1]  Gedare Bloom,et al.  Towards a provenance collection framework for Internet of Things devices , 2017, 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[2]  Edoardo Pignotti,et al.  Interrogating Capabilities of IoT Devices , 2014, IPAW.

[3]  Abigail Sellen,et al.  "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents , 2016, CHI.

[4]  Benjamin R. Cowan,et al.  "What can i help you with?": infrequent users' experiences of intelligent personal assistants , 2017, MobileHCI.

[5]  Tom Rodden,et al.  Provenance for the People: An HCI Perspective on the W3C PROV Standard through an Online Game , 2015, CHI.

[6]  Melanie Herschel,et al.  A survey on provenance: What for? What form? What from? , 2017, The VLDB Journal.

[7]  Matthew B Hoy Alexa, Siri, Cortana, and More: An Introduction to Voice Assistants , 2018, Medical reference services quarterly.

[8]  Qi Wang,et al.  Fear and Logging in the Internet of Things , 2018, NDSS.

[9]  Paul T. Groth,et al.  The provenance of electronic data , 2008, CACM.

[10]  Qi Li,et al.  Building accountability into the Internet of Things: the IoT Databox model , 2018, Journal of Reliable Intelligent Environments.

[11]  Biplab Sikdar,et al.  Secure Data Provenance for the Internet of Things , 2017, IoTPTS@AsiaCCS.

[12]  Paolo Missier,et al.  ProvGen: generating synthetic PROV graphs with predictable structure , 2014, IPAW.

[13]  Andreas Schreiber,et al.  Visualizing Provenance using Comics , 2017, TaPP.