Using think-aloud protocols in formative evaluation of interactive voice response systems

Interactive voice response (IVR) systems, which require people to respond on a telephone keypad to a series of aural prompts, are becoming increasingly widespread. Because writing or reading information is quite different from listening to it, guidance based on testing with actual users is particularly crucial to successful development of a usable IVR system.Think-aloud protocols, in which users verbalize their thoughts while interacting with an IVR script to perform specified tasks, provide an effective and economical assessment of the target audience's ability to use a proposed system. Based on the performance and comments of a small number of users, major problems with comprehension and usability can be corrected early in the development process, when it is easiest to make changes. Think-aloud protocols may also help us to better understand and anticipate the kinds of problems that users typically encounter with IVR systems, thus helping us to avoid some of these problems in future development efforts.Based on several recent think-aloud tests of IVR systems, this paper examines the use of think-aloud protocols at early stages of IVR system development. It includes an outline of methodology, concrete examples of results, and a brief examination of advantages and limitations of this method.