Towards the design of effective formative test reports

Many usability practitioners conduct most of their usability evaluations to improve a product during its design and development. We call these "formative" evaluations to distinguish them from "summative" (validation) usability tests at the end of development. A standard for reporting summative usability test results has been adopted by international standards organizations. But that standard is not intended for the broader range of techniques and business contexts in formative work. This paper reports on a new industry project to identify best practices in reports of formative usability evaluations. The initial work focused on gathering examples of reports used in a variety of business contexts. We define elements in these reports and present some early guidelines on making design decisions for a formative report. These guidelines are based on considerations of the business context, the relationship between author and audience, the questions that the evaluation is trying to answer, and the techniques used in the evaluation. Future work will continue to investigate industry practice and conduct evaluations of proposed guidelines or templates.