The Meaning and Measurement of User Satisfaction: A Multigroup Invariance Analysis of the End-User Computing Satisfaction Instrument

Although user satisfaction is widely used by researchers and practitioners to evaluate information system success, important issues related to its meaning and measurement across population subgroups have not been adequately resolved. To be most useful in decision-making, instruments like end-user computing satisfaction (EUCS), which are designed to evaluate system success, should be robust. That is, they should enable comparisons by providing equivalent measurement across diverse samples that represent the variety of conditions or population subgroups present in organizations. Using a sample of 1,166 responses, the EUCS instrument is tested for measurement invariance across four dimensions--respondent positions, types of application, hardware platforms, and modes of development. While the results suggest that the meaning of user satisfaction is context sensitive and differs across population subgroups, the 12 measurement items are invariant across all four dimensions. The 12-item summed scale enables researchers or practitioners to compare EUCS scores across the instrument's originally intended universe of applicability.

[1]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[2]  Charles A. Gallagher.  Perceptions of the Value of a Management Information System , 1974 .

[3]  David W. Conrath,et al.  What is being done to measure user satisfaction with EDP/MIS , 1990, Inf. Manag..

[4]  R. Gorsuch.  A Comparison of Biquartimin, Maxplane, Promax, and Varimax , 1970 .

[5]  Timothy Paul Cronan,et al.  On the Repeated Test-Retest Reliability of the End-User Computing Satisfaction Instrument: A Comment* , 1994 .

[6]  Magid Igbaria,et al.  End-user effectiveness: A cross-cultural examination , 1991 .

[7]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[8]  P. Bentler.  On the fit of models to covariances and methodology to the Bulletin. , 1992, Psychological bulletin.

[9]  Weidong Xia,et al.  Confirmatory Factor Analysis of the End-User Computing Satisfaction Instrument: A Replication , 1997 .

[10]  Arun Rai,et al.  Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis , 2002, Inf. Syst. Res..

[11]  Timothy Paul Cronan,et al.  Computer Simulation Success: On the Use of the End-User Computing Satisfaction Instrument: A Comment , 1998 .

[12]  Dale Goodhue,et al.  Supporting users of corporate data : the effect of I/S policy choices , 1988 .

[13]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[14]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[15]  Joseph O. Rentz.  Generalizability Theory: A Comprehensive Method for Assessing and Improving the Dependability of Marketing Measures , 1987 .

[16]  B. Byrne.  Structural Equation Modeling with LISREL, PRELIS, and SIMPLIS: Basic Concepts, Applications, and Programming , 1998 .

[17]  William J. Doll,et al.  The Measurement of End-User Computing Satisfaction , 1988, MIS Q..

[18]  E. B. Swanson,et al.  Management Information Systems: Appreciation and Involvement , 1974 .

[19]  H. Marsh.  The Factorial Invariance of Responses by Males and Females to a Multidimensional Self-Concept Instrument: Substantive and Methodological Issues. , 1987, Multivariate behavioral research.

[20]  Anthony R. Hendrickson,et al.  Using Davis's Perceived Usefulness and Ease-of-use Instruments for Decision Making: A Confirmatory and Multigroup Invariance Analysis , 1998 .

[21]  Maarten Gelderman,et al.  The relation between user satisfaction, usage of information systems and performance , 1998, Inf. Manag..

[22]  William J. Doll,et al.  Test‐Retest Reliability of the End‐User Computing Satisfaction Instrument , 1991 .

[23]  Ali F. Farhoomand,et al.  A structural model of end user computing satisfaction and user performance , 1996, Inf. Manag..

[24]  Suzanne Rivard,et al.  Factors of success for end-user computing , 1988, CACM.

[25]  Anol Bhattacherjee,et al.  Understanding Information Systems Continuance: An Expectation-Confirmation Model , 2001, MIS Q..

[26]  Wanda J. Orlikowski,et al.  A Short Form Measure of User Information Satisfaction: A Psychometric Evaluation and Notes on Use , 1987, J. Manag. Inf. Syst..

[27]  Barbara M. Byrne,et al.  Adolescent Self-Concept: Testing the Assumption of Equivalent Structure Across Gender , 1987 .

[28]  Robert W. Bailey,et al.  Human performance engineering: A guide for system designers , 1982 .

[29]  W. Buxton.  Human-Computer Interaction , 1988, Springer Berlin Heidelberg.

[30]  Benjamin B. M. Shao,et al.  The relationship between user participation and system success: a simultaneous contingency approach , 2000, Inf. Manag..

[31]  Sree Nilakanta,et al.  Measurement: A blueprint for theory-building in MIS , 1994, Inf. Manag..

[32]  Charles L. Hulin,et al.  The measurement of satisfaction in work and retirement: A strategy for the study of attitudes. , 1969 .

[33]  R. P. McDonald,et al.  Goodness-of-fit indexes in confirmatory factor analysis : The effect of sample size , 1988 .

[34]  P. Bentler,et al.  Significance Tests and Goodness of Fit in the Analysis of Covariance Structures , 1980 .

[35]  H. Marsh.  The Structure of Masculinity/Femininity: An Application of Confirmatory Factor Analysis to Higher-Order Factor Structures and Factorial Invariance. , 1985, Multivariate behavioral research.

[36]  Dale Goodhue,et al.  Task-Technology Fit and Individual Performance , 1995, MIS Q..

[37]  Roger McHaney,et al.  A validation of the end-user computing satisfaction instrument in Taiwan , 2002, Inf. Manag..

[38]  J. S. Long.  Confirmatory Factor Analysis: A Preface to Lisrel , 1983 .

[39]  Robert M. Alloway,et al.  User Managers' Systems Needs , 2015, MIS Q..

[40]  Fatemeh Zahedi,et al.  The Measurement of Web-Customer Satisfaction: An Expectation and Disconfirmation Approach , 2002, Inf. Syst. Res..

[41]  Maarten Gelderman,et al.  Task difficulty, task variability and satisfaction with management support systems , 2002, Inf. Manag..

[42]  W. Doll,et al.  A discrepancy model of end-user computing involvement , 1989 .

[43]  R. Kanfer,et al.  Equivalence of psychological measurement in heterogeneous populations. , 1985, The Journal of applied psychology.

[44]  Roger McHaney,et al.  EUCS test-retest reliability in representational model decision support systems , 1999, Inf. Manag..

[45]  Amy W. Gatian.  Is user satisfaction a valid measure of system effectiveness? , 1994, Inf. Manag..

[46]  Steven M. Lalonde,et al.  Multivariate Statistical Methods: A First Course , 2001, Technometrics.

[47]  Blake Ives,et al.  User Involvement and MIS Success: A Review of Research , 1984 .

[48]  Robert W. Zmud,et al.  Confounding Effects of Construct Overlap , 1994 .

[49]  Barbara M. Byrne,et al.  Strategies in testing for an invariant second‐order factor structure: A comparison of EQS and LISREL , 1995 .

[50]  L. Hayduk.  Structural equation modeling with LISREL , 1987 .

[51]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[52]  Vladimir Zwass,et al.  Foundations of information systems , 1997 .

[53]  Roger McHaney,et al.  Validation of the End-User Computing Satisfaction Instrument in Case Tool Environments , 2000, J. Comput. Inf. Syst..

[54]  Gina J. Medsker,et al.  A Review of Current Practices for Evaluating Causal Models in Organizational Behavior and Human Resources Management Research , 1994 .

[55]  Gary Klein,et al.  A new view of IS personnel performance evaluation , 2001, CACM.

[56]  Karin Klenke,et al.  Construct Measurement in Management Information systems: A Review and Critique of User Satisfaction , 1992 .

[57]  Isaac I. Bejar,et al.  Biased assessment of program impact due to psychometric artifacts. , 1980 .

[58]  Louis Raymond,et al.  Validating and applying user satisfaction as a measure of MIS success in small organizations , 1987, Inf. Manag..

[59]  Timothy Paul Cronan,et al.  Toward an empirical understanding of computer simulation implementation success , 2000, Inf. Manag..

[60]  Wynne W. Chin,et al.  Research Report - The Importance of Specification in Causal Modeling: The Case of End-User Computing Satisfaction , 1995, Inf. Syst. Res..

[61]  Magid Igbaria,et al.  End-user computing effectiveness: A structural equation model , 1990 .

[62]  Bill C. Hardgrave,et al.  Toward a Contingency Model for Selecting an Information System Prototyping Strategy , 1999, J. Manag. Inf. Syst..

[63]  N. Melone.  A theoretical assessment of the user-satisfaction construct in information systems research , 1990 .

[64]  Steven L. Alter.  Development Patterns for Decision Support Systems , 1978, MIS Q..

[65]  James F. Courtney,et al.  A Field Study of Organizational Factors Influencing DSS Success , 1985, MIS Q..

[66]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[67]  Louis Raymond,et al.  Development of a measure to assess the quality of user-developed applications , 1997, DATB.

[68]  R. Bagozzi,et al.  On the evaluation of structural equation models , 1988 .

[69]  Holmes Miller,et al.  The Multiple Dimensions of Information Quality , 1996, Inf. Syst. Manag..

[70]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[71]  William J. Doll,et al.  A Confirmatory Factor Analysis of the End-User Computing Satisfaction Instrument , 1994, MIS Q..

[72]  Steven L. Alter,et al.  How Effective Managers Use Information Systems. , 1976 .

[73]  H. Marsh.  Confirmatory factor analysis models of factorial invariance: A multifaceted approach , 1994 .

[74]  G Larry Sanders,et al.  MIS/DSS success measure , 1984 .

[75]  Tor Guimaraes,et al.  Successful Strategies for User Participation in Systems Development , 1997, J. Manag. Inf. Syst..

[76]  J. Schaubroeck,et al.  Confirmatory Modeling in Organizational Behavior/Human Resource Management: Issues and Applications , 1990 .

[77]  Simha R. Magal,et al.  Determinants of Information Center Success , 1998, J. Manag. Inf. Syst..

[78]  H. Marsh,et al.  Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. , 1985 .

[79]  L. Tucker,et al.  A reliability coefficient for maximum likelihood factor analysis , 1973 .

[80]  B. Wheaton,et al.  Assessment of Fit in Overidentified Models with Latent Variables , 1987 .