How usability testing can aid the development of online documentation

Over the past five years, usability testing has become a more accepted part of the software development process as writers, managers, and programmers recognize the need for evaluating their products from the user's point of view. As the application of usability testing has grown, so has the interest in procedures and methods. One indication of that is the number of presentations on usability testing at conferences and the increased number of publications on usability testing. Last year, for instance, an entire issue of the IEEE Transactions on Technical Communications devoted to usability testing. As Ramey (1989a) explains in that issue, “the question of whether to has yielded to the question of how to” (p. 207). The interest in procedures and methods has also brought about a concern for applying usability testing not just to validate computer products but also to provide feedback to product developers (Sullivan, 1987). In fact, theorists and practitioners of usability testing argue that iterative testing (that is, testing a product frequently during its development cycle) is the most productive application of usability testing (Dieli, 1989a). By testing iteratively, usability testing can have the greatest effect on the development of the product. One problem, of course, is finding methods that can be used to test products at each stage of the development process. For example, how can writers usability test an online help system before a working prototype has been created? And how do they test semi-functional prototypes? The problem is especially acute for developers of online documentation. In general, the development process of online documentation contains more variables than print documentation, which means that more things can go wrong. For instance, online must be integrated with a product, yet it may be developed separately and “put together” near the end of the development cycle. In addition, development can be slower, because online depends on several technologies (the text, the help engine, perhaps multimedia), whereas print documentation generally depends on one. The testing “toolbox” therefore needs to be especially responsive to a product's developmental stages. In this article I describe how testing methodologies can be used to test online help systems iteratively. Specifically, I discuss a framework for visualizing usability testing of online documentation through the development cycle. But first, two limitations on the scope of the article. The discussion is intended for writers and online developers who are new to usability testing and for students and teachers in documentation writing courses, although experienced researchers may find it helpful as a planning aid. Second, the article focuses on exploratory or “feedback” kinds of tests, rather than tests that confirm or validate products. Before going any further, let me define what I mean by “online documentation.” Horton (1990), in his textbook on designing online documentation, explains that online documentation can encompass a number of forms, from on-screen error messages to computer-based tutorials. My discussion will be restricted to two kinds of online documentation. The first is what often is called “help”—basically a reference document online. In many of the Microsoft Help systems, for instance, users can find information on program commands (analogous to a reference manual) as well as information about procedures (analogous to a user's manual). The other type of online documentation is an online tutorial, computer-based training (CBT). The tutorial describes basic procedures and commands and it interactive so that there is a major “hands-on” component to the training.