Towards Fine-Grained Usability Testing: New Methodological Directions with Conversation Analysis

We examine the possibilities of conversation analysis (CA) in usability testing. The goal is to examine how usability test setups serve as source for the CA analysis. We used video data from two earlier usability tests. Our results indicate that traditional test setup does not serve as a sufficient source for CA. The actions in the user interface were unclear, the user's facial reactions were not visible, and the user is occasionally having more conversation with the moderator than the system. CA approach can be taken towards two separate directions regarding usability tests: Analysis can be focused to the dialogue between the moderator and the user or on the user-system interaction. There is a need to fine-tune data gathering with detailed level recording of keypresses and system outputs. However, CA-enhanced usability testing allows in-depth analysis of usability problems as well as analysis of holistic interaction between user and system.

[1]  Randolph G. Bias,et al.  Cost-justifying usability , 2005 .

[2]  Allison Woodruff,et al.  Conversation analysis and the user experience , 2004, Digit. Creativity.

[3]  L. Suchman.  Human-Machine Reconfigurations: Plans and situated actions (2nd edition). , 2007 .

[4]  Pirkko Liisa Raudaskoski.  Repair Work in Human-Computer Interaction , 1990 .

[5]  Wes Sharrock,et al.  On simulacrums of conversation: toward a clarification of the relevance of conversation analysis for human-computer interaction , 1995 .

[6]  David Frohlich,et al.  Computers and conversation , 1990 .

[7]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[8]  Lucy Suchman,et al.  Human-Machine Reconfigurations: Plans and Situated Actions , 2006 .

[9]  Paul ten Have,et al.  Doing Conversation Analysis: A Practical Guide , 1999 .

[10]  Joseph S. Dumas,et al.  Moderating Usability Tests: Principles and Practices for Interacting: Principles and Practices for Interacting , 2008 .

[11]  J. M. Atkinson.  Structures of Social Action: Contents , 1985 .

[12]  C. Goodwin.  Action and embodiment within situated human interaction , 2000 .

[13]  Marko Nieminen,et al.  Launching Usability Approach at Helsinki University of Technology , 1995 .

[14]  Pamela J. Hinds,et al.  Workplace Studies: Recovering Work Practice and Informing System Design, Paul Luff, Jon Hindmarsh and Christian C. Heath (eds.) , 2000, Computer Supported Cooperative Work (CSCW).

[15]  P. Thomas.  The social and interactional dimensions of human-computer interfaces , 1995 .

[16]  Sirpa Riihiaho,et al.  Launching the usability approach: experience at Helsinki University of Technology , 1995, SGCH.