Development of a Heuristic Evaluation Tool for Voice User Interfaces

Voice user interfaces (VUIs) are now a common means of interaction with IT systems. To assist in evaluating the usability of such interfaces, a set of evaluation heuristics assessing speech interfaces was developed by following an existing methodology for defining usability heuristics. Two groups of eight participants conducted an evaluation by inspection of three speech-based systems, a mobile phone assistant, a smart speaker and an in-car hands free phone system. One group used Nielsen and Molich’s general heuristics for user interface design while the other group used the VUI heuristics. The second group found, on average, more problems than the first group. However, most heuristics from both sets were rated as useful during the study. This indicates that a mixture of both general and application specific heuristics are needed for a comprehensive evaluation to be performed. Experience from a pilot study, where a smart speaker was set up in a domestic setting, highlighted the need to also consider social and environmental issues to gain a complete picture of user experience when interacting with speech systems.