Towards a combined method of web usability testing: an assessment of the complementary advantages of lab testing, pre-session assignments, and online usability services

Lab-based testing is one of the key methods employed for evaluating web site usability. Yet the artificial conditions of the setting, including surveillance and stylized tasks, can distort user behavior and limit the data that can be obtained. This paper reports on the effectiveness of this standard method compared against two complementary methods which involve more natural, user-driven evaluation contexts, namely, pre-session homework assignments and online usability testing. Using illustrations from recent studies of online shopping sites we detail the advantages and limitations of each method and claim that employing them in combination could improve the quantity and quality of findings. We then propose that future work should focus on optimizing this combined method through sequencing, so that one evaluation approach would inform the design of subsequently used ones.