Towards Understanding How Agile Teams Predict User Experience

In this chapter, we compare UX assessments of users and agile team members to learn to what extent developers can predict how users experience (UX) the product the developers are working on, and where user involvement is truly required. We compared UX assessments of agile team members (N = 26) and users (N = 29) of six enterprise applications with statistical tests. Moreover, we analyzed the data with principal component analysis to reveal the main dimensions of UX for enterprise software. Our results confirm prior research findings that agile team members can put themselves in the users’ position when evaluating instrumental aspects of UX of the software they are working on. However, it seems that developers cannot evaluate non-instrumental quality. Therefore, direct user involvement from participation to evaluation or other means to support user empathy in development process is needed. We recommend additional means, such as personas to help agile team members empathize with the users and their needs for non-instrumental qualities of the enterprise software.

[1]  Heli Väätäjä,et al.  The Fuzzy Front End of Experience Design: Eliciting and Communicating Experience Goals , 2015, OZCHI.

[2]  Sarah Diefenbach,et al.  The 'hedonic' in human-computer interaction: history, contributions, and future research directions , 2014, Conference on Designing Interactive Systems.

[3]  Kati Kuusinen,et al.  Task Allocation Between UX Specialists and Developers in Agile Software Development Projects , 2015, INTERACT.

[4]  Brian Shackel,et al.  Usability - Context, framework, definition, design and evaluation , 1991, Interact. Comput..

[5]  Brian Fitzgerald,et al.  Continuous software engineering and beyond: trends and challenges , 2014, RCoSE 2014.

[6]  Susan M. Dray,et al.  Remote possibilities?: international usability testing at a distance , 2004, INTR.

[7]  Marc Hassenzahl,et al.  The Thing and I: Understanding the Relationship Between User and Product , 2005, Funology.

[8]  Virpi Roto,et al.  Tracing links between UX frameworks and design practices: dual carriageway , 2014 .

[9]  H. Marsh,et al.  Development and Validation of a Scale to Measure Optimal Experience: The Flow State Scale , 1996 .

[10]  Fred D. Davis.  A technology acceptance model for empirically testing new end-user information systems : theory and results , 1985 .

[11]  Meenakshi Panwar.  Application Performance Management Emerging Trends , 2013, 2013 International Conference on Cloud & Ubiquitous Computing & Emerging Technologies.

[12]  Gilbert Cockton,et al.  Understanding Inspection Methods: Lessons from an Assessment of Heuristic Evaluation , 2001, BCS HCI/IHM.

[13]  Andreas Holzinger,et al.  Usability engineering methods for software developers , 2005, CACM.

[14]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[15]  Jurek Kirakowski,et al.  The Software Usability Measurement Inventory: Background and Usage , 1996 .

[16]  Anders Bruun,et al.  Let your users do the testing: a comparison of three remote asynchronous usability testing methods , 2009, CHI.

[17]  Virpi Roto,et al.  Defining user experience goals to guide the design of industrial systems , 2015, Behav. Inf. Technol..

[18]  Marc Hassenzahl,et al.  The Interplay of Beauty, Goodness, and Usability in Interactive Products , 2004, Hum. Comput. Interact..

[19]  Yanan Chen,et al.  Do Usability Professionals Think about User Experience in the Same Way as Users and Developers Do? , 2013, INTERACT.

[20]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[21]  Virpi Roto,et al.  User experience evaluation methods: current state and development needs , 2010, NordiCHI.

[22]  Tommi Mikkonen,et al.  On Designing UX for Mobile Enterprise Apps , 2014, 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications.

[23]  Hanna-Riikka Sundberg,et al.  The Role of User Experience in a Business-to-Business Context , 2015 .

[24]  Pieter Desmet,et al.  Designing Products with Added Emotional Value: Development and Appllcation of an Approach for Research through Design , 2001 .

[25]  Gitte Lindgaard,et al.  Introduction to the Special Issue: The Tricky Landscape of Developing Rating Scales in HCI , 2013, Interact. Comput..

[26]  Raj Gururajan,et al.  Expectations, usability, and job satisfaction as determinants for the perceived benefits for the use of wireless technology in healthcare , 2013 .

[27]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[28]  Ernest A. Edmonds,et al.  A study in play, pleasure and interaction design , 2007, DPPI.

[29]  John C. McCarthy,et al.  Technology as experience , 2004, INTR.

[30]  F.R.H. Zijlstra,et al.  Efficiency in work behaviour: A design approach for modern tools , 1993 .

[31]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[32]  J. M. Christian Bastien,et al.  Usability testing: a review of some methodological and technical aspects of the method , 2010, Int. J. Medical Informatics.

[33]  L. Godwin,et al.  Trends and challenges. , 1960, Hospital progress.

[34]  Vincent Koenig,et al.  User experience: A concept without consensus? Exploring practitioners' perspectives through an international survey , 2015, Comput. Hum. Behav..

[35]  J. Nunnally.  Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[36]  Kasper Hornbæk,et al.  Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience , 2011, CHI.

[37]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[38]  Jan Gulliksen,et al.  Informal feedback rather than performance measurements – user-centred evaluation in Scrum projects , 2014, Behav. Inf. Technol..

[39]  Morten Hertzum,et al.  Personal Usability Constructs: How People Construe Usability Across Nationalities and Stakeholder Groups , 2011, Int. J. Hum. Comput. Interact..

[40]  Marc Hassenzahl,et al.  User experience (UX): towards an experiential perspective on product quality , 2008, IHM '08.

[41]  Virpi Roto,et al.  Understanding, scoping and defining user experience: a survey approach , 2009, CHI.

[42]  Gerald Albaum,et al.  Is a Central Tendency Error Inherent in the Use of Semantic Differential Scales in Different Cultures? , 2003 .

[43]  Fethi Calisir,et al.  The relation of interface usability characteristics, perceived usefulness, and perceived ease of use to end-user satisfaction with enterprise resource planning (ERP) systems , 2004, Comput. Hum. Behav..

[44]  L GoodhueDale,et al.  Task-technology fit and individual performance , 1995 .

[45]  Noam Tractinsky,et al.  Assessing dimensions of perceived visual aesthetics of web sites , 2004 .

[46]  Desirée Sy,et al.  Adapting usability investigations for agile user-centered design , 2007 .

[47]  Dale Goodhue,et al.  Task-Technology Fit and Individual Performance , 1995, MIS Q..

[48]  Virpi Roto,et al.  Towards Practical User Experience Evaluation Methods , 2008 .

[49]  Kristopher J Preacher,et al.  Exploratory Factor Analysis in Behavior Genetics Research: Factor Recovery with Small Sample Sizes , 2002, Behavior genetics.

[50]  Anders Bruun,et al.  The effect of task assignments and instruction types on remote asynchronous usability testing , 2012, CHI.

[51]  Paul A. Cairns,et al.  A systematic literature review for agile development processes and user centred design integration , 2014, EASE '14.

[52]  Cheol Park,et al.  Mobile technology usage and B2B market performance under mandatory adoption , 2008 .

[53]  N. Blunch.  Position Bias in Multiple-Choice Questions , 1984 .

[54]  A. Mehrabian.  Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in Temperament , 1996 .

[55]  José Antonio Pow-Sang,et al.  A systematic review of usability techniques in agile methodologies , 2014, EATIS '14.

[56]  Kevin E. Voss,et al.  Measuring the Hedonic and Utilitarian Dimensions of Consumer Attitude , 2003 .

[57]  Danilo Caivano,et al.  Usability evaluation: a survey of software development organizations , 2011, SEKE.

[58]  Mary Poppendieck,et al.  Lean Software Development: An Agile Toolkit , 2003 .

[59]  Morten Hertzum,et al.  How do usability professionals construe usability? , 2012, Int. J. Hum. Comput. Stud..

[60]  Frank Maurer,et al.  User-Centered Design and Agile Methods: A Systematic Review , 2011, 2011 AGILE Conference.

[61]  Ken Schwaber,et al.  Agile Project Management with Scrum , 1980 .

[62]  H. Abdi,et al.  Principal component analysis , 2010 .

[63]  Virpi Roto,et al.  Developing practical tools for user experience evaluation: a case from mobile news journalism , 2009, ECCE.

[64]  Marti A. Hearst,et al.  The state of the art in automating usability evaluation of user interfaces , 2001, CSUR.

[65]  Jon Innes.  Why Enterprises Can't Innovate: Helping Companies Learn Design Thinking , 2011, HCI.

[66]  Hossein Saiedian,et al.  Requirements engineering: making the connection between the software developer and customer , 2000, Inf. Softw. Technol..