Construct Measurement in Management Information systems: A Review and Critique of User Satisfaction

AbstractConstruct measurement in management information systems (MIS) is still very much in its infancy and has not yet produced instruments of adequate reliability and validity, thereby raising questions about the robustness of substantive findings. User satisfaction (US) and user involvement (UI) are two central MIS constructs which have been conceptualized and operationalized in a number of different ways and from divergent theoretical perspectives. The most commonly used measures of US and UI are critically evaluated here against a set of traditional psychometric criteria. Based on this evaluation, a number of psychometric and substantive problems regarding the scales are identified which are categorized as scale, sample, reliability, methodological and reporting problems. A variety of methods and techniques designed to overcome the measurement problems analyzed in this research are discussed to increase the quality of construct measurement in MIS and recommendations for an ongoing program of coostruc...

[1]  David W. Conrath,et al.  What is being done to measure user satisfaction with EDP/MIS , 1990, Inf. Manag..

[2]  James A. Senn,et al.  Research in Management Information Systems: The Minnesota Experiments , 1977 .

[3]  Henri Barki,et al.  Change, attitude to change, and decision support system success , 1985, Inf. Manag..

[4]  Karl G. Jöreskog,et al.  Analyzing psychological data by structural analysis of covariance matrices , 1974 .

[5]  G. Dickson,et al.  Organizational Characteristics and Information Systems: An Exploratory Investigation , 1982 .

[6]  E. Burton Swanson,et al.  Research in Management Information Systems, 1980-1984: Points of Work and Reference , 1986, MIS Q..

[7]  J. Nunnally.  Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[8]  Joseph A. Cote,et al.  Lack of method variance in self-reported affect and perceptions at work: Reality or artifact? , 1989 .

[9]  H. Blalock.  Conceptualization and Measurement in the Social Sciences , 1982 .

[10]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[11]  N. Melone.  A theoretical assessment of the user-satisfaction construct in information systems research , 1990 .

[12]  Mary J. Culnan,et al.  Mapping the Intellectual Structure of MIS, 1980-1985: A Co-Citation Analysis , 1987, MIS Q..

[13]  Daniel W. Russell,et al.  Discriminant validation of measures of job satisfaction, job involvement, and organizational commitment. , 1988 .

[14]  Gerardine DeSanctis,et al.  ICIS Paper: Methodological Issues in Experimental IS Research: Experiences and Recommendations , 1985, MIS Q..

[15]  Detmar W. Straub,et al.  Validating Instruments in MIS Research , 1989, MIS Q..

[16]  Daniel Robey,et al.  User Involvement in Information System Development: A Conflict Model and Empirical Test , 1982 .

[17]  Enid Mumford,et al.  Designing systems for job satisfaction , 1973 .

[18]  John C. Henderson,et al.  An Evolutionary Strategy for Implementing a Decision Support System , 1981 .

[19]  Rob Kling,et al.  The organizational context of user-centered software designs* , 1977 .

[20]  Henry C. Lucas,et al.  Why information systems fail , 1975 .

[21]  Louis Raymond,et al.  Organizational Characteristics and MIS Success in the Context of Small Business , 1985, MIS Q..

[22]  D. Larcker,et al.  PERCEIVED USEFULNESS OF INFORMATION: A PSYCHOMETRIC EXAMINATION* , 1980 .

[23]  Henri Barki,et al.  Rethinking the Concept of User Involvement , 1989, MIS Q..

[24]  C. Schewe.  The Management Information System User: An Exploratory Behavioral Analysis , 1976 .

[25]  Suzanne Rivard,et al.  Factors of success for end-user computing , 1988, CACM.

[26]  Kailash Joshi,et al.  The Measurement of Fairness or Equality Perceptions of Management Information Systems Users , 1989, MIS Q..

[27]  C. R. Franz,et al.  ORGANIZATIONAL CONTEXT, USER INVOLVEMENT, AND THE USEFULNESS OF INFORMATION SYSTEMS* , 1986 .

[28]  Dennis F. Galletta,et al.  Some Cautions on the Measurement of User Information Satisfaction , 1989 .

[29]  Avi Rushinek,et al.  What makes users happy? , 1986, CACM.

[30]  Alexander M. Maish.  A User's Behavior Toward His MIS , 1979, MIS Q..

[31]  Blake Ives,et al.  User Involvement and MIS Success: A Review of Research , 1984 .

[32]  Iris Vessey,et al.  The Effect of User Involvement on System Success: A Contingency Approach , 1988, MIS Q..

[33]  Paul E. Spector.  Method variance as an artifact in self-reported affect and perceptions at work: Myth or significant problem? , 1987 .

[34]  Roger E. Millsap,et al.  A cautionary note on the detection of method variance in multitrait-multimethod data , 1990 .

[35]  H. Marsh,et al.  A new, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analysis. , 1988 .

[36]  Blake Ives,et al.  An empirical study of the impact of user involvement on system usage and information satisfaction , 1986, CACM.

[37]  R. N. Kanungo.  Work alienation: An integrative approach , 1983 .

[38]  W. Doll,et al.  A discrepancy model of end-user computing involvement , 1989 .

[39]  Ali R. Montazemi,et al.  Factors Affecting Information Satisfaction in the Context of the Small Business Environment , 1988, MIS Q..

[40]  Charles Russel Franz,et al.  Contingency factors affecting the user involvement role in the design of successful information systems. , 1979 .

[41]  E. A. Locke,et al.  Participation in Decision-Making: One More Look , 1979 .

[42]  K. Weick.  Theory Construction as Disciplined Imagination , 1989 .

[43]  G. Laurent,et al.  Measuring Consumer Involvement Profiles , 1985 .

[44]  Blake Ives,et al.  User Involvement in System Design: An Empirical Test of Alternative Approaches , 1981, Inf. Manag..

[45]  Gary I. Green,et al.  Perceived Importance of Systems Analysts' Job Skills, Roles, and Non-Salary Incentives , 1989, MIS Q..

[46]  Ann-Marie K. Baronas,et al.  Restoring a Sense of Control During Implementation: How User Involvement Leads to System Acceptance , 1988, MIS Q..

[47]  E. B. Swanson,et al.  Management Information Systems: Appreciation and Involvement , 1974 .

[48]  Howard B. Lee,et al.  Foundations of Behavioral Research , 1973 .

[49]  William J. Doll,et al.  The Measurement of End-User Computing Satisfaction , 1988, MIS Q..

[50]  Fritz Drasgow,et al.  Psychometric and substantive issues in scale construction and validation , 1982 .

[51]  Tan Boon Wan,et al.  Validation of a user satisfaction instrument for office automation success , 1990, Inf. Manag..

[52]  K. Widaman.  Hierarchically Nested Covariance Structure Models for Multitrait-Multimethod Data , 1985 .

[53]  Wanda J. Orlikowski,et al.  A Short Form Measure of User Information Satisfaction: A Psychometric Evaluation and Notes on Use , 1987, J. Manag. Inf. Syst..

[54]  Anders Edstrom,et al.  User Influence and the Success of MIS Projects: A Contingency Approach , 1977 .

[55]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[56]  Michael J. Kavanagh,et al.  Issues in managerial performance: Multitrait-multimethod analyses of ratings. , 1971 .

[57]  Zur Shapira,et al.  On the use of facet analysis in organizational behavior research: Some conceptual considerations and an example , 1979 .

[58]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[59]  William R. King,et al.  Note-Participative Design of Strategic Decision Support Systems: An Empirical Assessment , 1981 .

[60]  William H. DeLone.  Determinants of success for small business computer systems , 1983 .