User experience evaluation methods: current state and development needs

The recent shift of emphasis to user experience (UX) has rendered it a central focus of product design and evaluation. A multitude of methods for UX design and evaluation exist, but a clear overview of the current state of the available UX evaluation methods is missing. This is partly due to a lack of agreement on the essential characteristics of UX. In this paper, we present the results of our multi-year effort of collecting UX evaluation methods from academia and industry with different approaches such as literature review, workshops, Special Interest Groups sessions and an online survey. We have collected 96 methods and analyzed them, among other criteria, based on the product development phase and the studied period of experience. Our analysis reveals development needs for UX evaluation methods, such as early-stage methods, methods for social and collaborative UX evaluation, establishing practicability and scientific quality, and a deeper understanding of UX.

[1]  Katherine Isbister,et al.  Chapter 5 – Let the Game Tester Do the Talking: Think Aloud and Interviewing to Learn About the Game Experience , 2008 .

[2]  Shailey Minocha,et al.  User Interface Design and Evaluation (The Morgan Kaufmann Series in Interactive Technologies) (The Morgan Kaufmann Series in Interactive Technologies) , 2005 .

[3]  Virpi Roto,et al.  User experience evaluation: do you know which method to use? , 2009, CHI Extended Abstracts.

[4]  William W. Gaver,et al.  Design: Cultural probes , 1999, INTR.

[5]  Marc Hassenzahl,et al.  User experience (UX): towards an experiential perspective on product quality , 2008, IHM '08.

[6]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[7]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[8]  Virpi Roto,et al.  User Experience Evaluation Methods in Product Development (UXEM'09) , 2009, INTERACT.

[9]  Kaisa Väänänen-Vainio-Mattila,et al.  Value of Information Systems and Products: Understanding the Users’ Perspective and Values , 2009 .

[10]  Jarmo Laaksolahti,et al.  Evaluating experience-focused HCI , 2007, CHI Extended Abstracts.

[11]  M. Tscheligi,et al.  Applying Psychophysiological Methods for Measuring User Experience : Possibilities , Challenges and Feasibility , 2009 .

[12]  Gilbert Cockton,et al.  An investigation into the use of field methods in the design and evaluation of interactive systems , 2008, BCS HCI.

[13]  Regan L. Mandryk,et al.  Measuring affect in hci: going beyond the individual , 2008, CHI Extended Abstracts.

[14]  M. Isomursu.  User experience evaluation with experimental pilots , 2008 .

[15]  Virpi Roto,et al.  Understanding, scoping and defining user experience: a survey approach , 2009, CHI.

[16]  John Zimmerman,et al.  User experience over time: an initial framework , 2009, CHI.

[17]  Minna Isomursu,et al.  Experimental evaluation of five methods for collecting emotions in field settings with mobile applications , 2007, Int. J. Hum. Comput. Stud..

[18]  K. Lewin,et al.  Field Theory in Social Science: Selected Theoretical Papers , 1951 .

[19]  Andreas Pfeiffer,et al.  User experience research , 2006, UBIQ.