End-User Development of Voice User Interfaces Based on Web Content

Voice Assistants, and particularly the latest gadgets called smart speakers, allow end users to interact with applications by means of voice commands. As usual, end users are able to install applications (also called skills) that are available in repositories and fulfill multiple purposes. In this work we present an end-user environment to define skills for voice assistants based on the extraction of Web content and their organization into different voice navigation patterns. We describe the approach, the end-user development environment, and finally we present some case studies based on Alexa and Amazon Echo.

[1]  Gustavo Rossi,et al.  Abstracting and Structuring Web Contents for Supporting Personal Web Experiences , 2016, ICWE.

[2]  Minos N. Garofalakis,et al.  MashMaker: mashups for the masses , 2007, SIGMOD '07.

[3]  Gustavo Rossi,et al.  From Search Engines to Augmented Search Services: An End-User Development Approach , 2017, ICWE.

[4]  Johanna Völker,et al.  Deployment of RDFa, Microdata, and Microformats on the Web - A Quantitative Analysis , 2013, International Semantic Web Conference.

[5]  Mary Shaw,et al.  The state of the art in end-user software engineering , 2011, ACM Comput. Surv..

[6]  Yuan Tian,et al.  Understanding and Mitigating the Security Risks of Voice-Controlled Third-Party Skills on Amazon Alexa and Google Home , 2018, ArXiv.

[7]  Frank Bentley,et al.  Understanding the Long-Term Use of Smart Speaker Assistants , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[8]  Claude Sammut,et al.  InCA: A Mobile Conversational Agent , 2004, PRICAI.

[9]  Ryen W. White.  Skill discovery in virtual assistants , 2018, Commun. ACM.

[10]  Xavier Le Pallec,et al.  Multimodal interaction: a survey from model driven engineering and mobile perspectives , 2013, Journal on Multimodal User Interfaces.

[11]  K. Chang,et al.  Embodiment in conversational interfaces: Rea , 1999, CHI '99.

[12]  Sapna George,et al.  Personalized voice command systems in multi modal user interface , 2012, 2012 IEEE International Conference on Emerging Signal Processing Applications.

[13]  Hamid Shahnasser,et al.  Internet of Things using Node-Red and alexa , 2017, 2017 17th International Symposium on Communications and Information Technologies (ISCIT).

[14]  Pasquale De Meo,et al.  Web Data Extraction , Applications and Techniques : A Survey , 2010 .

[15]  Marco Winckler,et al.  Web Augmentation as a Promising Technology for End User Development , 2017, New Perspectives in End-User Development.

[16]  J. C. R. Licklider,et al.  Man-Computer Symbiosis , 1960 .

[17]  Tantek Çelik,et al.  Microformats: a pragmatic path to the semantic web , 2006, WWW '06.

[18]  Gustavo Rossi,et al.  End-User Software Engineering for the Personal Web , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C).

[19]  Oscar Díaz,et al.  Addressing web locator fragility: a case for browser extensions , 2017, EICS.

[20]  Jordi Cabot,et al.  Model-Driven Software Engineering in Practice , 2017, Synthesis Lectures on Software Engineering.

[21]  David R. Karger,et al.  Atomate it! end-user context-sensitive automation using heterogeneous information sources on the web , 2010, WWW '10.