Using the End-User Computing Satisfaction (EUCS) Instrument to Measure Satisfaction with a Web Site

The purpose of this study is to revise and revalidate the End-User Computing Satisfaction (EUCS) instrument to measure satisfaction with a Web site from a usability perspective. This study is especially important given the increased significance of the Web and the uniqueness of the Web as a computing environment. A total of 176 students participated in a lab simulation that involved a usability evaluation of the Lands' End Web site (http://www.landsend.com). Students were asked to complete a set of tasks, record their answers, and then complete the EUCS instrument. Confirmatory factor analysis and invariance analyses were conducted to test the reliability, validity, and generalizability of the revised EUCS. The results show that the EUCS is a valid and robust instrument in the Web environment but that one of the subfactors, timeliness, will need further refinement in the future. Usability practitioners can use the EUCS to measure end-user satisfaction with a Web site and use the feedback for improving Web-site design. We describe a case study of an actual usability application that utilized the revised EUCS effectively to support the design of building supply Web sites involving two types of end users, homeowners and contractors. We also propose a typology that researchers can use as a starting point to judge when it is necessary to revalidate an instrument like the EUCS. Finally, we discuss the limitations of our study and present avenues for future research.

[1]  Timothy Paul Cronan,et al.  On the Repeated Test-Retest Reliability of the End-User Computing Satisfaction Instrument: A Comment* , 1994 .

[2]  Young-Gul Kim,et al.  Extending the TAM for a World-Wide-Web context , 2000, Inf. Manag..

[3]  Ke-Hai Yuan,et al.  F Tests for Mean and Covariance Structure Analysis , 1999 .

[4]  Hanhwe Kim,et al.  Spatial metaphors and disorientation in hypertext browsing , 1995, Behav. Inf. Technol..

[5]  Detmar W. Straub,et al.  Validation in Information Systems Research: A State-of-the-Art Assessment , 2001, MIS Q..

[6]  Marios Koufaris,et al.  Applying the Technology Acceptance Model and Flow Theory to Online Consumer Behavior , 2002, Inf. Syst. Res..

[7]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[8]  Stephanie Rosenbaum,et al.  Choosing Usability Research Methods , 2000 .

[9]  Eric W.T. Ngai,et al.  A critical review of end-user information system satisfaction research and a new research framework , 2002 .

[10]  J. Jaccard,et al.  LISREL Approaches to Interaction Effects in Multiple Regression , 1998 .

[11]  Peter M. Bentler,et al.  EQS : structural equations program manual , 1989 .

[12]  Mary Corbett,et al.  SUMI: the Software Usability Measurement Inventory , 1993, Br. J. Educ. Technol..

[13]  K. Mardia.  Measures of multivariate skewness and kurtosis with applications , 1970 .

[14]  William J. Doll,et al.  Test‐Retest Reliability of the End‐User Computing Satisfaction Instrument , 1991 .

[15]  Timothy Paul Cronan,et al.  Computer Simulation Success: On the Use of the End-User Computing Satisfaction Instrument: A Comment , 1998 .

[16]  Kent L. Norman,et al.  The effect of tree structure on search in a hierarchical menu selection system , 1988 .

[17]  P. Bentler.  Comparative Fit Indices in Structural Models , 1990 .

[18]  Jonathan W. Palmer,et al.  Web Site Usability, Design, and Performance Metrics , 2002, Inf. Syst. Res..

[19]  H. Han,et al.  Is It Worth It to Target University Students , 2002 .

[20]  Abigail Sellen,et al.  How knowledge workers use the web , 2002, CHI.

[21]  Karin Klenke,et al.  Construct Measurement in Management Information systems: A Review and Critique of User Satisfaction , 1992 .

[22]  Detmar W. Straub,et al.  Measuring e-Commerce in Net-Enabled Organizations: An Introduction to the Special Issue , 2002, Inf. Syst. Res..

[23]  Mary Czerwinski,et al.  Organizing usability work to fit the full product range , 1999, CACM.

[24]  Fatemeh Zahedi,et al.  The Measurement of Web-Customer Satisfaction: An Expectation and Disconfirmation Approach , 2002, Inf. Syst. Res..

[25]  Jakob Nielsen,et al.  Designing Web Usability: The Practice of Simplicity , 1999 .

[26]  Bradley C. Wheeler,et al.  NEBIC: A Dynamic Capabilities Theory for Assessing Net-Enablement , 2002, Inf. Syst. Res..

[27]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[28]  Mo Adam Mahmood,et al.  Factors Affecting Information Technology Usage: A Meta-Analysis of the Empirical Literature , 2001, J. Organ. Comput. Electron. Commer..

[29]  Anne P. Massey,et al.  From the user interface to the consumer interface: results from a global experiment , 2000, Int. J. Hum. Comput. Stud..

[30]  J. Greenberg,et al.  Organizational behavior : the state of the science , 1994 .

[31]  Roger McHaney,et al.  EUCS test-retest reliability in representational model decision support systems , 1999, Inf. Manag..

[32]  M. Aldenderfer,et al.  Cluster Analysis. Sage University Paper Series On Quantitative Applications in the Social Sciences 07-044 , 1984 .

[33]  Mo Adam Mahmood,et al.  Variables affecting information technology end-user satisfaction: a meta-analysis of the empirical literature , 2000, Int. J. Hum. Comput. Stud..

[34]  David Kaplan,et al.  Statistical power in structural equation modeling. , 1995 .

[35]  Jurek Kirakowski,et al.  Measuring the Usability of Web Sites , 1998 .

[36]  J. S. Long,et al.  Testing Structural Equation Models , 1993 .

[37]  Bernard Baruch,et al.  Measuring e-Commerce in Net-Enabled Organizations: An Introduction to the Special Issue , 2002 .

[38]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[39]  J. S. Long,et al.  Testing Structural Equation Models , 1993 .

[40]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[41]  Debora Shaw,et al.  Handbook of usability testing: How to plan, design, and conduct effective tests , 1996 .

[42]  Weidong Xia,et al.  Confirmatory Factor Analysis of the End-User Computing Satisfaction Instrument: A Replication , 1997 .

[43]  Judy Chuan-Chuan Lin,et al.  Towards an understanding of the behavioural intention to use a web site , 2000, Int. J. Inf. Manag..

[44]  Asaf Zohar,et al.  Refining Our Understanding of Hypercompetition and Hyperturbulence , 1996 .

[45]  Fiona Fui-Hoon Nah,et al.  HCI Research Issues in Electronic Commerce , 2002, J. Electron. Commer. Res..

[46]  Donna L. Hoffman,et al.  A New Marketing Paradigm for Electronic Commerce , 1997, Inf. Soc..

[47]  Viswanath Venkatesh,et al.  Assessing a Firm's Web Presence: A Heuristic Evaluation Procedure for the Measurement of Usability , 2002, Inf. Syst. Res..

[48]  Ulman Lindenberger,et al.  Statistical Power in Structural Equation Modeling , 2007 .

[49]  Albert L. Lederer,et al.  The technology acceptance model and the World Wide Web , 2000, Decis. Support Syst..

[50]  Detmar W. Straub,et al.  Research Commentary: Transformational Issues in Researching IS and Net-Enabled Organizations , 2001, Inf. Syst. Res..

[51]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[52]  Karim Alrawi.  The internet and international marketing , 2007 .

[53]  William J. Doll,et al.  A Confirmatory Factor Analysis of the End-User Computing Satisfaction Instrument , 1994, MIS Q..

[54]  William J. Doll,et al.  The Measurement of End-User Computing Satisfaction , 1988, MIS Q..

[55]  H. Rex Hartson,et al.  Human-computer interaction: Interdisciplinary roots and trends , 1998, J. Syst. Softw..

[56]  Rajiv Kohli,et al.  Antecedents of B2C Channel Satisfaction and Preference: Validating e-Commerce Metrics , 2002, Inf. Syst. Res..

[57]  Seymour Sudman,et al.  Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research , 1995 .

[58]  Daniel Cunliffe,et al.  Developing usable Web sites - a review and model , 2000, Internet Res..

[59]  Janice Redish,et al.  User and task analysis for interface design , 1998 .

[60]  W. Bogner,et al.  Making Sense in Hypercompetitive Environments: A Cognitive Explanation for the Persistence of High Velocity Competition , 2000 .