A Fast Form Approach to Measuring Technology Acceptance and Other Constructs

Nearly all prior studies on the technology acceptance model (TAM) have used Likert scales to measure the model's constructs, but the use of only this type of scale has two shortcomings. One is that such use prevents us from exposing the model's constructs to a robust test of their measure and relationships to each other, termed their nomological validity. The other is that such use leaves us unsure about whether or not we have selected an efficient way, in terms of survey completion time, to assess these constructs. Past researchers have used short form scales to address the issue of efficiency, but there are problems that may result from such efforts. In this study, we address both shortcomings by exploring the use of a semantic differential scale, which we refer to as a fast form, to assess the constructs of TAM. In this regard, we do three things. First, we describe how fast form as a scale may be developed. Second, we conduct a psychometric evaluation of the constructs that are measured by the fast form and examine their relationships. Third, we assess the efficiency of the fast form by comparing the time required to complete a survey with it to that which is required to complete a survey with Likert scales. Our results confirm that the constructs that are measured by the fast form are psychometrically equivalent to those that are measured by the Likert scales. The relationship among these constructs was unchanged, providing strong evidence for nomological validity. The fast form also yielded a 40 percent reduction in the survey completion time, proving its superior efficiency. We conclude with a description of the implications of these results for research and practice.

[1]  Monica Martinussen,et al.  Likert-based vs. semantic differential-based scorings of positive psychological constructs: A psychometric comparison of two versions of a scale measuring resilience. , 2006 .

[2]  B. Byrne,et al.  Multigroup Comparisons and the Assumption of Equivalent Construct Validity Across Groups: Methodological and Substantive Issues. , 1989, Multivariate behavioral research.

[3]  M. Kendall,et al.  The Logic of Scientific Discovery. , 1959 .

[4]  Detmar W. Straub,et al.  Veni, Vidi, Vici: Breaking the TAM Logjam , 2007, J. Assoc. Inf. Syst..

[5]  Patrick M. Morgan,et al.  Deterrence Now: Testing, testing, one … two … three , 2003 .

[6]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[7]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[8]  P. Bentler,et al.  Covariance structure analysis: statistical practice, theory, and directions. , 1996, Annual review of psychology.

[9]  Izak Benbasat,et al.  Quo vadis TAM? , 2007, J. Assoc. Inf. Syst..

[10]  Wanda J. Orlikowski,et al.  A Short Form Measure of User Information Satisfaction: A Psychometric Evaluation and Notes on Use , 1987, J. Manag. Inf. Syst..

[11]  Robert C. MacCallum,et al.  Effect of Estimation Method on Incremental Fit Indexes for Covariance Structure Models , 1993 .

[12]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[13]  Wynne W. Chin.  The partial least squares approach for structural equation modeling. , 1998 .

[14]  Timothy Paul Cronan,et al.  On the Test-Retest Reliability of Perceived Usefulness and Perceived Ease of Use Scales , 1993 .

[15]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[16]  S Flamer,et al.  Assessment of the Multitrait-Multimethod Matrix Validity of Likert Scales Via Confirmatory Factor Analysis. , 1983, Multivariate behavioral research.

[17]  Younghwa Lee,et al.  The Technology Acceptance Model: Past, Present, and Future , 2003, Commun. Assoc. Inf. Syst..

[18]  J. P. Peter.  Reliability: A Review of Psychometric Basics and Recent Marketing Practices , 1979 .

[19]  R. Hoyle.  Structural equation modeling: concepts, issues, and applications , 1997 .

[20]  K. Popper,et al.  The Logic of Scientific Discovery , 1960 .

[21]  L. Hayduk,et al.  Testing! testing! one, two, three - Testing the theory in structural equation models! , 2007 .

[22]  P. Bentler,et al.  Evaluating model fit. , 1995 .

[23]  Robert F. DeVellis,et al.  Scale Development: Theory and Applications. , 1992 .

[24]  M. Sussmann,et al.  The validity of validity: An analysis of validation study designs. , 1986 .

[25]  H W Marsh,et al.  Positive and negative global self-esteem: a substantively meaningful distinction or artifactors? , 1996, Journal of personality and social psychology.

[26]  Gi Woong Yun,et al.  Comparative Response to a Survey Executed by Post, E-mail, & Web Form , 2006, J. Comput. Mediat. Commun..

[27]  R. MacCallum,et al.  Power analysis and determination of sample size for covariance structure modeling. , 1996 .

[28]  Craig J. Russell,et al.  Moderated Regression Analysis and Likert Scales: Too Coarse for Comfort , 1992 .

[29]  R. Motl,et al.  Longitudinal Invariance of Self-Esteem and Method Effects Associated With Negatively Worded Items , 2002 .

[30]  Detmar W. Straub,et al.  Validation Guidelines for IS Positivist Research , 2004, Commun. Assoc. Inf. Syst..

[31]  J. Nunnally.  Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[32]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[33]  J. Dawson,et al.  On the Use of Likert-Type Scales in Multilevel Data , 2007 .

[34]  R. Peterson.  A Meta-analysis of Cronbach's Coefficient Alpha , 1994 .

[35]  Peter A. Todd,et al.  Perceived Usefulness, Ease of Use, and Usage of Information Technology: A Replication , 1992, MIS Q..

[36]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[37]  Dale N. Glaser,et al.  Jiving the Four-Step, Waltzing Around Factor Analysis, and Other Serious Fun , 2000 .

[38]  C. Osgood,et al.  The Measurement of Meaning , 1958 .

[39]  H. Marsh,et al.  In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler's (1999) Findings , 2004 .

[40]  Fred D. Davis,et al.  A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies , 2000, Management Science.

[41]  Gilbert A. Churchill,et al.  Relationships among research design choices and psychometric properties of rating scales: A meta-analysis. , 1986 .

[42]  Detmar W. Straub,et al.  Validation in Information Systems Research: A State-of-the-Art Assessment , 2001, MIS Q..

[43]  S. West,et al.  The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. , 1996 .

[44]  I. Ajzen,et al.  Understanding Attitudes and Predicting Social Behavior , 1980 .

[45]  R. Kline.  Principles and practice of structural equation modeling, 2nd ed. , 2005 .

[46]  K. Jöreskog,et al.  Intraclass Reliability Estimates: Testing Structural Assumptions , 1974 .

[47]  Thomas J. Johnson,et al.  Research Methodology: Taming the Cyber Frontier , 1999 .

[48]  Gilbert A. Churchill,et al.  Research Design Effects on the Reliability of Rating Scales: A Meta-Analysis: , 1984 .

[49]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[50]  Samuel Shye,et al.  Introduction to Facet Theory: Content Design and Intrinsic Data Analysis in Behavioral Research , 1994 .

[51]  Gerald Albaum,et al.  The measurement properties of semantic scale data. , 1977 .

[52]  Peter M. Bentler,et al.  On tests and indices for evaluating structural models , 2007 .

[53]  Kristen G. Anderson,et al.  On the sins of short-form development. , 2000, Psychological assessment.

[54]  Barbara M. Byrne,et al.  Structural equation modeling with EQS : basic concepts, applications, and programming , 2000 .

[55]  Francis J. Yammarino,et al.  UNDERSTANDING MAIL SURVEY RESPONSE BEHAVIOR , 2004 .

[56]  Wynne W. Chin,et al.  Looking Forward: Toward an Understanding of the Nature and Definition of IT Acceptance , 2007, J. Assoc. Inf. Syst..

[57]  Daniel J. Mueller.  Measuring Social Attitudes: A Handbook for Researchers and Practitioners , 1986 .

[58]  N. Schwarz.  Cognition and Communication , 2004, Inner Worlds.

[59]  Detmar W. Straub,et al.  Measuring System Usage: Implications for IS Theory Testing , 1995 .

[60]  J. Overall,et al.  The Brief Psychiatric Rating Scale , 1962 .

[61]  K. Bollen,et al.  An Empirical Evaluation of the Use of Fixed Cutoff Points in RMSEA Test Statistic in Structural Equation Models , 2008, Sociological methods & research.

[62]  Samuel Shye,et al.  Introduction to Facet Theory , 1994 .

[63]  I. Borg,et al.  Facet Theory: Form and Content , 1995 .

[64]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .