Analysis of the Persuasiveness of User Experience Feedback on a Virtual Learning Environment

The main purpose of this paper is to evaluate a set of theory-grounded User Experience (UX)-related measures which are supposed to persuade course designers of particular UX-related problem areas based on a specific feedback format. Specifically, two online surveys on a university online course were conducted with the former focusing on the quantitative ratings and the latter on qualitative comments. The course designers were asked to assess the persuasiveness of the feedback with respect to eight dimensions. The results show that UX-related problem areas anchored in the DeLone and McLean’s Information Systems (IS) Success Model (ISSM) had a consistently higher level of perceived persuasiveness than those anchored in the Technology Acceptance Model (TAM) 3. The discrepancy can be attributed to the nature of items: processvs. trait-based). Implications for future research on fixing UX-related problems are discussed.

[1]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[2]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[3]  Gitte Lindgaard.  Usability testing and system evaluation - a guide for designing useful computer systems , 1994, Chapman and Hall computing series.

[4]  Kar Yan Tam,et al.  Determinants of User Acceptance of Digital Libraries: An Empirical Examination of Individual Differences and System Characteristics , 2002, J. Manag. Inf. Syst..

[5]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[6]  D. Sandy Staples,et al.  Dimensions of Information Systems Success , 1999, Commun. Assoc. Inf. Syst..

[7]  Viswanath Venkatesh,et al.  Determinants of Perceived Ease of Use: Integrating Control, Intrinsic Motivation, and Emotion into the Technology Acceptance Model , 2000, Inf. Syst. Res..

[8]  Fred D. Davis,et al.  A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies , 2000, Management Science.

[9]  I. Ajzen,et al.  Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research , 1977 .

[10]  Nigel Bevan,et al.  What is the difference between the purpose of usability and user experience evaluation methods , 2009 .

[11]  Stefan Strohmeier,et al.  Developing and Validating a Rigorous and Relevant Model of VLE Success: A Learner Perspective , 2010, EC-TEL.

[12]  Sergio L. Toral Marín,et al.  A technological acceptance of e-learning tools used in practical and laboratory teaching, according to the European higher education area 1 , 2008, Behav. Inf. Technol..

[13]  Marc Hassenzahl,et al.  From mental effort to perceived usability: transforming experiences into summary assessments , 2004, CHI EA '04.

[14]  Stefan Strohmeier,et al.  Design Characteristics of Virtual Learning Environments: An Expert Study , 2010 .

[15]  A. Adam.  Whatever happened to information systems ethics? Caught between the devil and the deep blue sea , 2004 .

[16]  Effie Lai-Chong Law.  Evaluating the Downstream Utility of User Tests and Examining the Developer Effect: A Case Study , 2006, Int. J. Hum. Comput. Interact..

[17]  Paul van Schaik,et al.  Modelling user experience - An agenda for research and practice , 2010, Interact. Comput..

[18]  Viswanath Venkatesh,et al.  Technology Acceptance Model 3 and a Research Agenda on Interventions , 2008, Decis. Sci..

[19]  Virpi Roto,et al.  Understanding, scoping and defining user experience: a survey approach , 2009, CHI.

[20]  Barbara H Wixom,et al.  A Theoretical Integration of User Satisfaction and Technology Acceptance , 2005, Inf. Syst. Res..

[21]  Daniel Mueller,et al.  A Learner-Centered Design, Implementation, and Evaluation Approach of Learning Environments to Foster Acceptance , 2009, Int. J. Adv. Corp. Learn..

[22]  Kasper Hornbæk,et al.  Exploring the Value of Usability Feedback Formats , 2009, Int. J. Hum. Comput. Interact..

[23]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[24]  Dennis R. Wixon,et al.  Making a difference—the impact of inspections , 1996, CHI.

[25]  Morten Hertzum,et al.  Problem Prioritization in Usability Evaluation: From Severity Assessments Toward Impact on Design , 2006, Int. J. Hum. Comput. Interact..

[26]  Marta Larusdottir.  Measuring the User Experience of a Task Oriented Software , 2008 .

[27]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[28]  Martin Bichler,et al.  Design science in information systems research , 2006, Wirtschaftsinf..

[29]  Peter A. Todd,et al.  Understanding Information Technology Usage: A Test of Competing Models , 1995, Inf. Syst. Res..

[30]  Marc Hassenzahl,et al.  The Interplay of Beauty, Goodness, and Usability in Interactive Products , 2004, Hum. Comput. Interact..

[31]  Joseph J. Martocchio,et al.  Microcomputer playfulness: development of a measure with workplace implications , 1992 .

[32]  Sheng-Chin Yu,et al.  Activate Interaction Relationships Between Students Acceptance Behavior and E-Learning , 2007, APWeb/WAIM.

[33]  Ephraim R. McLean,et al.  Measuring information systems success: models, dimensions, measures, and interrelationships , 2008, Eur. J. Inf. Syst..

[34]  Izak Benbasat,et al.  Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation , 1991, Inf. Syst. Res..

[35]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..