Measuring System Usage: Implications for IS Theory Testing

There is widespread agreement among researchers that system usage, defined as the utilization of information technology IT by individuals, groups, or organizations, is the primary variable through which IT affects white collar performance. Despite the number of studies targeted at explaining system usage, there are crucial differences in the way the variable has been conceptualized and operationalized. This wide variation of system usage measures hinders the efforts of MIS researchers to compare findings across studies, thus impeding the accumulation of knowledge and theory in this area. The purpose of this paper is to address conceptual as well as methodological issues related to measuring system usage. First, via LISREL measurement modeling techniques, we compare subjective and objective measures of system usage, namely, self-reported versus computer-recorded measures. Next, using a modified form of Davis' Technology Acceptance Model TAM as a nomological net, we test the nomological validity of these system usage constructs and measures. Results of the LISREL measurement and nomological net analysis suggest that system usage should be factored into self-reported system usage and computer-recorded system usage. Contrary to expectations, these constructs do not appear to be strongly related to each other. Moreover, while self-reported measures of system usage are related to self-reported measures of TAM independent variables, objective, computer-recorded measures show distinctly weaker links. In the face of such counter-evidence, it is tempting to argue that research that has relied on subjective measures of system usage for example, research confirming TAM may be artifactual. There are several alternative explanations, though, that maintain the integrity of TAM and studies that measure system usage subjectively. These alternative explanations suggest directions for further research as well as new approaches to measurement.

[1]  Paul H. Cheney,et al.  Concepts, Theory, and Techniques: FACTORS AFFECTING THE PERCEIVED UTILIZATION OF COMPUTER-BASED DECISION SUPPORT SYSTEMS IN THE OIL INDUSTRY , 1982 .

[2]  C. Steinfield.  Dimensions of Electronic Mail Use in an Organizational Setting. , 1985 .

[3]  T. Connolly,et al.  Information mispurchase in judgment tasks: A task-driven causal mechanism , 1988 .

[4]  R. Zmud.  INDIVIDUAL DIFFERENCES AND MIS SUCCESS: A REVIEW OF THE EMPIRICAL LITERATURE* , 1979 .

[5]  François Bergeron,et al.  Factors Influencing the Use of DP Chargeback Information , 1986, MIS Q..

[6]  Detmar W. Straub,et al.  Key information liability issues facing managers: software piracy, proprietary databases, and indi , 1990 .

[7]  I. Ajzen.  Attitude structure and behavior. , 1989 .

[8]  J. Pfeffer,et al.  A social information processing approach to job attitudes and task design. , 1978, Administrative science quarterly.

[9]  C. Schewe.  The Management Information System User: An Exploratory Behavioral Analysis , 1976 .

[10]  Gordon B. Davis,et al.  Management information systems : conceptual foundations, structure, and development , 1985 .

[11]  Ronald E. Rice,et al.  The use of computer-monitored data in information science and communication research , 1983, J. Am. Soc. Inf. Sci..

[12]  Izak Benbasat,et al.  Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation , 1991, Inf. Syst. Res..

[13]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[14]  E. B. Swanson,et al.  Measuring user attitudes in MIS research: a review , 1982 .

[15]  R. Gooding,et al.  Effects of Societal Trends on Participation Research. , 1987 .

[16]  Louis Raymond,et al.  Organizational Characteristics and MIS Success in the Context of Small Business , 1985, MIS Q..

[17]  Jack T. Hogue.  A Framework for the Examination of Management Involvement in Decision Support Systems , 1987, J. Manag. Inf. Syst..

[18]  James S. Ettema,et al.  Explaining Information System Use with System-Monitored vs. Self-Reported Use Measures , 1985 .

[19]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[20]  William R. King,et al.  Note-Participative Design of Strategic Decision Support Systems: An Empirical Assessment , 1981 .

[21]  M. Treacy,et al.  Utilization as a dependent variable in MIS research , 2015, DATB.

[22]  Dale Goodhue,et al.  Supporting users of corporate data : the effect of I/S policy choices , 1988 .

[23]  E. Burton Swanson,et al.  INFORMATION CHANNEL DISPOSITION AND USE , 1987 .

[24]  Brian K. Thorn,et al.  Predecisional information acquisition: Effects of task variables on suboptimal search strategies , 1987 .

[25]  Jinjoo Lee,et al.  An exploratory contingency model of user participation and MIS use , 1986, Inf. Manag..

[26]  G. D. Weeks,et al.  Subjective time estimates of work tasks by office workers , 1977 .

[27]  Detmar W. Straub,et al.  Information technologies for the 1990s: an organizational impact perspective , 1989, CACM.

[28]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[29]  Lee Sproull,et al.  Reducing social context cues: electronic mail in organizational communication , 1986 .

[30]  P. Bentler,et al.  Significance Tests and Goodness of Fit in the Analysis of Covariance Structures , 1980 .

[31]  Alexander M. Maish.  A User's Behavior Toward His MIS , 1979, MIS Q..

[32]  Ronald E. Rice,et al.  Electronic Message Systems in the University: A Description of Use and Utility , 1983 .

[33]  Hugh J. Parry,et al.  Validity of Responses to Survey Questions , 1950 .

[34]  Gordon B. Davis,et al.  A Framework for Research in Computer-Based Management Information Systems , 1980 .

[35]  N. L. Reinsch,et al.  Voice Mail Versus Conventional Channels: A Cost Minimization Analysis of Individuals' Preferences , 1990 .

[36]  James Jaccard,et al.  An informational approach to altruistic behavior. , 1976 .

[37]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[38]  E. B. Swanson,et al.  Management Information Systems: Appreciation and Involvement , 1974 .

[39]  William H. DeLone.  Determinants of Success for Computer Usage in Small Business , 1988, MIS Q..

[40]  Jane M. Howell,et al.  Personal Computing: Toward a Conceptual Model of Utilization , 1991, MIS Q..

[41]  I. Ajzen,et al.  Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research , 1977 .

[42]  Gerry Gingrich,et al.  Subjective evaluation and allocation of resources in routine decision making , 1984 .

[43]  Kieran Mathieson,et al.  Predicting User Intentions: Comparing the Technology Acceptance Model with the Theory of Planned Behavior , 1991, Inf. Syst. Res..

[44]  A. Tversky,et al.  Judgment under Uncertainty , 1982 .

[45]  KieslerSara,et al.  Reducing Social Context Cues , 1986 .

[46]  Phillip Ein-Dor,et al.  Use of Management Information Systems: An Empirical Study , 1981, ICIS.

[47]  Daniel Robey,et al.  User Attitudes and Management Information System Use , 1979 .

[48]  D. Bem.  Self-Perception Theory , 1972 .

[49]  Robin M. Hogarth,et al.  Cognitive Processes and the Assessment of Subjective Probability Distributions , 1975 .

[50]  Ananth Srinivasan,et al.  Alternative Measures of Systems Effectiveness: Associations and Implications , 1985, MIS Q..

[51]  H C Triandis,et al.  Values, attitudes, and interpersonal behavior. , 1980, Nebraska Symposium on Motivation. Nebraska Symposium on Motivation.

[52]  Scott Hamilton,et al.  Evaluating Information Systems Effectiveness - Part I: Comparing Evaluation Approaches , 1981, MIS Q..

[53]  Michael J. Ginzberg,et al.  Early Diagnosis of MIS Implementation Failure: Promising Results and Unanswered Questions , 1981 .

[54]  Detmar W. Straub,et al.  Validating Instruments in MIS Research , 1989, MIS Q..

[55]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[56]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[57]  Henry C. Lucas,et al.  A descriptive model of information systems in the context of the organization , 1973, DATB.

[58]  Ramesh Sharda,et al.  Decision support system effectiveness: a review and an empirical test , 1988 .

[59]  Peter A. Todd,et al.  Perceived Usefulness, Ease of Use, and Usage of Information Technology: A Replication , 1992, MIS Q..

[60]  Fred D. Davis.  A technology acceptance model for empirically testing new end-user information systems : theory and results , 1985 .

[61]  M. Lynne Markus,et al.  Toward a “Critical Mass” Theory of Interactive Media , 1987 .

[62]  Dale L. Goodhue,et al.  User evaluations of MIS success: what are we really measuring? , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.