UX Evaluation Methodology for iTV: Assessing a Natural Language Interaction System

The user experience can be evaluated in different ways, from a combination of approaches and techniques capable of collecting insights about a particular product or system. In the TV ecosystem research field, verifying how users react to new features is an essential step to the integration of functionalities able to enhance the user experience. This is the case of systems based on interaction by natural language (NLI), which have the potential to allow for more simplified navigation (based on conversational dynamics). However, although the spoken interactions are relevant to optimize the consumption of television content, it is essential to identify how they are received and understanded by the user. In this context, this empirical study sought to analyze the experience of using an NLI system controlled by a mobile application. The evaluation was performed employing an open methodology that considers instrumental and non-instrumental qualities of the application, as well as emotional dimensions. This approach was specifically developed for UX analysis of systems and applications related to the TV ecosystem, having recorded positive results in previous studies. In the study here presented, the methodology revealed again to be suitable as it was possible to identify failures and opportunities to improve the assessed NLI system and, finally, to verify that the voice interaction system allowed users a more optimized and accessible experience.