A Systematic Review About User Experience Evaluation

The user experience related to a product can determine its success or failure. As a result, companies have invested in research to understand what the user feels when he or she uses a product, but it is not simple to get this information, and many approaches arose through the years. This article presents a systematic review on the user experience evaluation field, based on 25 studies. We found (1) that psychophysiological measures are not yet widely applied in the evaluation of the user experience; (2) researchers prefer qualitative approaches; (3) the evaluations are mostly manual (not real time, therefore), and using ready-made products; and (4) most studies used single measurement, not considering the use of the product over time. This scenario shows an opportunity for UX evaluation methods which consider the use of the product over time and provide real time results.

[1]  Anshu Agarwal,et al.  Beyond usability: evaluating emotional response as an integral part of the user experience , 2009, CHI Extended Abstracts.

[2]  Paul van Schaik,et al.  Attitudes towards user experience (UX) measurement , 2014, Int. J. Hum. Comput. Stud..

[3]  Fabio Abbattista,et al.  User brain-driven evaluation of an educational 3D virtual environment , 2015, 2015 10th Iberian Conference on Information Systems and Technologies (CISTI).

[4]  Marc Hassenzahl,et al.  The Interplay of Beauty, Goodness, and Usability in Interactive Products , 2004, Hum. Comput. Interact..

[5]  Darelle van Greunen,et al.  User experience evaluation metrics for usable accounting tools , 2010, SAICSIT '10.

[6]  Martin Schrepp,et al.  Research-in-progress: User experience evaluation of Student Centered E-Learning Environment for computer science program , 2014, 2014 3rd International Conference on User Science and Engineering (i-USEr).

[7]  Sascha Mahlke,et al.  Studying antecedents of emotional experiences in interactive contexts , 2007, CHI.

[8]  Aiur Retegi,et al.  Eyeface: A new multimethod tool to evaluate the perception of conceptual user experiences , 2015, Comput. Hum. Behav..

[9]  Virpi Roto,et al.  User Experience Evaluation Methods in Academic and Industrial Contexts , 2009 .

[10]  Lei Zhou,et al.  Using Physiological Measures to Evaluate User Experience of Mobile Applications , 2014, HCI.

[11]  Monica Landoni,et al.  A diary study of children's user experience with EBooks using flow theory as framework , 2014, IDC.

[12]  Gavin Sim,et al.  Evaluating game preference using the fun toolkit across cultures , 2012, BCS HCI.

[13]  Virpi Roto,et al.  UX Curve: A method for evaluating long-term user experience , 2011, Interact. Comput..

[14]  Vaninha Vieira,et al.  Um estudo experimental de avaliação da experiência dos usuários de aplicativos móveis a partir da captura automática dos dados contextuais e de interação , 2012, IHC.

[15]  Gavin Sim,et al.  Investigating children's opinions of games: Fun Toolkit vs. This or That , 2012, IDC '12.

[16]  Gavin Sim,et al.  Investigating the fidelity effect when evaluating game prototypes with children , 2013, BCS HCI.

[17]  Manfred Tscheligi,et al.  MAESTRO: orchestrating user behavior driven and context triggered experience sampling , 2010, MB '10.

[18]  Kaisa Väänänen,et al.  Let users tell the story: evaluating user experience with experience reports , 2010, CHI Extended Abstracts.

[19]  Markku Turunen,et al.  Evaluating the experiential user experience of public display applications in the wild , 2013, MUM.

[20]  Helen Petrie,et al.  Axe UX: Exploring long-term user experience with iScale and AttrakDiff , 2014, MindTrek.

[21]  Eric N. Wiebe,et al.  Measuring engagement in video game-based environments: Investigation of the User Engagement Scale , 2014, Comput. Hum. Behav..

[22]  Cédric Bach,et al.  Combining interviews and scales in the multidimensional evaluation of user experience: a case study in 3D games , 2011, ECCE.

[23]  Annemarie Lesage,et al.  Designing and Evaluating User Experience , 2017 .

[24]  Markku Tukiainen,et al.  Evaluating user experience of autistic children through video observation , 2013, CHI Extended Abstracts.

[25]  Kaisa Väänänen,et al.  Little backpackers: studying children's psychological needs in an interactive exhibition context , 2013, IDC.

[26]  Toni Granollers,et al.  OPEN-HEREDEUX: OPEN HEuristic REsource for Designing and Evaluating User eXperience , 2011, INTERACT.

[27]  Janet C. Read.  Evaluating artefacts with children: age and technology effects in the reporting of expected and experienced fun , 2012, ICMI '12.

[28]  J. H. Gilmore,et al.  Welcome to the experience economy. , 1998, Harvard business review.

[29]  Virpi Roto,et al.  Identifying hedonic factors in long-term user experience , 2011, DPPI.

[30]  M. P. Cota,et al.  Measurement of user experience: A Spanish language version of the user experience questionnaire (UEQ) , 2012, 7th Iberian Conference on Information Systems and Technologies (CISTI 2012).

[31]  Virpi Roto,et al.  Understanding, scoping and defining user experience: a survey approach , 2009, CHI.

[32]  Tanja Mercun.  Evaluation of information visualization techniques: analysing user experience with reaction cards , 2014, BELIV '14.

[33]  Nicu Sebe,et al.  UX_Mate: from facial expressions to UX evaluation , 2012, DIS '12.

[34]  Matthias Peissner,et al.  From product concept to user experience: exploring UX potentials at early product stages , 2010, NordiCHI.

[35]  Virpi Roto,et al.  User experience evaluation: do you know which method to use? , 2009, CHI Extended Abstracts.

[36]  Heidi Krömker,et al.  A framework to measure user experience of interactive online products , 2010, MB '10.

[37]  Federico Chesani,et al.  User Experience (UX) of the Fall Risk Assessment Tool (FRAT-up) , 2015, 2015 IEEE 28th International Symposium on Computer-Based Medical Systems.

[38]  Foglia Pierfrancesco,et al.  Towards Relating Physiological Signals to Usability Metrics: A Case Study with a Web Avatar , 2014 .

[39]  Gao Changyuan,et al.  Research on user experience evaluation system of information platform based on web environment , 2013, Proceedings of 2013 2nd International Conference on Measurement, Information and Control.