The Efficiency of Various Multimodal Input Interfaces Evaluated in Two Empirical Studies

SUMMARY Although research into multimodal interfaces has been around for a long time, we believe that some basic issues have not been studied yet, e.g. the choice of modalities and their combinations is usually made without any quantitative evaluation. This study seeks to identify the best combinations of modalities through usability testing. How do users choose different interaction modes when they work on a particular application? Two experimental evaluations were conducted to compare interaction modes on a CAD system and a map system respectively. For the CAD system, the results show that, in terms of total manipulation time (drawing and modification time) and subjective preferences, the “pen + speech + mouse” combination was the best of the seven interaction modes tested. On the map system, the results show that the “pen + speech” combination mode is the best of fourteen interaction modes tested. The experiments also provide information on how users adapt to each interaction mode and the ease with which they are able to use these modes.