The test-retest reliability of user involvement instruments

Abstract It is almost an axiom of the MIS literature that user involvement is a necessary condition for successful development of information systems. Despite this acceptance, Ives and Olson have observed that, hampered by conflicting findings and weak measures, over two decades of research has not convincingly demonstrated the benefits of user involvement. They contend that there is a need to develop reliable and valid measures of user involvement and system success. This paper reports on the extends a program to develop standard user involvement instruments that can measure end-user developed applications as well as others using more traditional methodology. The short-range and long-range stability of instruments that measure percieved involvement, desired involvement, and involvement congruence is assessed via the test-retest method. The results suggest that these instruments are internally consistent, stable, and can be used with confidence in user involvement research without concern about a reactivity effect.

[1]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[2]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[3]  Wanda J. Orlikowski,et al.  The Problem of Statistical Power in MIS Research , 1989, MIS Q..

[4]  N. M. Downie,et al.  Descriptive and Inferential Statistics , 2019, Companion Encyclopedia of Psychology.

[5]  B. Brabander,et al.  Successful Information System Development in Relation to Situational Factors Which Affect Effective Communication Between MIS-Users and EDP-Specialists , 1984 .

[6]  Aaron Lowin,et al.  Participative decision making: A model, literature critique, and prescriptions for research , 1968 .

[7]  William J. Doll,et al.  A Congruence Construct of User Involvement , 1991 .

[8]  E. B. Swanson,et al.  Management Information Systems: Appreciation and Involvement , 1974 .

[9]  William J. Doll,et al.  Test‐Retest Reliability of the End‐User Computing Satisfaction Instrument , 1991 .

[10]  Anders Edstrom,et al.  User Influence and the Success of MIS Projects: A Contingency Approach , 1977 .

[11]  E. A. Locke,et al.  Participation in Decision-Making: One More Look , 1979 .

[12]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[13]  Anders Edstrom,et al.  Successful Information System Development Projects , 1977 .

[14]  B. Wilpert,et al.  Conceptual dimensions and boundaries of participation in organizations: a critical evaluation. , 1978, Administrative science quarterly.

[15]  Suzanne Rivard,et al.  Factors of success for end-user computing , 1988, CACM.

[16]  C. R. Franz,et al.  ORGANIZATIONAL CONTEXT, USER INVOLVEMENT, AND THE USEFULNESS OF INFORMATION SYSTEMS* , 1986 .

[17]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[18]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.

[19]  William J. Doll,et al.  The measurement of end-user software involvement , 1990 .

[20]  Detmar W. Straub,et al.  Validating Instruments in MIS Research , 1989, MIS Q..

[21]  Lucas Hc.  The evolution of an information system: from key-man to every person. , 1978 .

[22]  Daniel Robey,et al.  User Involvement in Information System Development: A Conflict Model and Empirical Test , 1982 .

[23]  W. Doll,et al.  A discrepancy model of end-user computing involvement , 1989 .

[24]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[25]  Blake Ives,et al.  User Involvement in System Design: An Empirical Test of Alternative Approaches , 1981, Inf. Manag..

[26]  Blake Ives,et al.  User Involvement and MIS Success: A Review of Research , 1984 .

[27]  William J. Doll,et al.  The Measurement of End-User Computing Satisfaction , 1988, MIS Q..

[28]  D. Vollrath,et al.  Employee Participation: Diverse Forms and Different Outcomes , 1988 .

[29]  Dennis F. Galletta,et al.  Some Cautions on the Measurement of User Information Satisfaction , 1989 .