Estimating the effect of common method variance: the method-method pair technique with an illustration from TAM research

This paper presents a meta-analysis-based technique to estimate the effect of common method variance on the validity of individual theories. The technique explains between-study variance in observed correlations as a function of the susceptibility to common method variance of the methods employed in individual studies. The technique extends to mono-method studies the concept of method variability underpinning the classic multitrait-multimethod technique. The application of the technique is demonstrated by analyzing the effect of common method variance on the observed correlations between perceived usefulness and usage in the technology acceptance model literature. Implications of the technique and the findings for future research are discussed.

[1]  Magid Igbaria,et al.  A Motivational Model of Microcomputer Usage , 1996, J. Manag. Inf. Syst..

[2]  A. Adam.  Whatever happened to information systems ethics? Caught between the devil and the deep blue sea , 2004 .

[3]  Ulrike Gretzel,et al.  Tourism Managers’ Adoption of Marketing Decision Support Systems , 2000 .

[4]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[5]  N. Venkatraman,et al.  Measurement of Business Economic Performance: An Examination of Method Convergence , 1987 .

[6]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[7]  T. Dickinson,et al.  A comparison of the behaviorally anchored rating and mixed standard scale formats. , 1980 .

[8]  Detmar W. Straub,et al.  Trust and TAM in Online Shopping: An Integrated Model , 2003, MIS Q..

[9]  Detmar W. Straub,et al.  Gender Differences in the Perception and Use of E-Mail: An Extension to the Technology Acceptance Model , 1997, MIS Q..

[10]  Rajeev Sharma,et al.  Common Methods Bias: Reports of Its Death are Greatly Exaggerated , 2007, ICIS.

[11]  Amy B. Woszczynski,et al.  The Problem of Common Method Variance in IS Research , 2004 .

[12]  Izak Benbasat,et al.  Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation , 1991, Inf. Syst. Res..

[13]  Elazar J. Pedhazur,et al.  Measurement, Design, and Analysis: An Integrated Approach , 1994 .

[14]  Kevin B. Lowe,et al.  Effectiveness correlates of transformational and transactional leadership: A meta-analytic review of the mlq literature , 1996 .

[15]  Sanford Labovitz,et al.  The Assignment of Numbers to Rank Order Categories , 1970 .

[16]  Fred D. Davis,et al.  A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies , 2000, Management Science.

[17]  Martin T. Wells,et al.  USING RANDOM RATHER THAN FIXED EFFECTS MODELS IN META‐ANALYSIS: IMPLICATIONS FOR SITUATIONAL SPECIFICITY AND VALIDITY GENERALIZATION , 1996 .

[18]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[19]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[20]  William R. King,et al.  Association for Information Systems (AIS) , 2010 .

[21]  Marcia J. Simmering,et al.  A Tale of Three Perspectives , 2009 .

[22]  Rajeev Sharma,et al.  The Contingent Effects of Training, Technical Complexity, and Task Interdependence on Successful Information Systems Implementation , 2007, MIS Q..

[23]  Beverly M. Calkins DHSc.  A meta-analysis of the role of smoking in inflammatory bowel disease , 2005, Digestive Diseases and Sciences.

[24]  Izak Benbasat,et al.  Quo vadis TAM? , 2007, J. Assoc. Inf. Syst..

[25]  RaiArun,et al.  Assessing the Validity of IS Success Models , 2002 .

[26]  William H. Glick,et al.  Common Methods Bias: Does Common Methods Variance Really Bias Results? , 1998 .

[27]  Bernadette Szajna,et al.  Empirical evaluation of the revised technology acceptance model , 1996 .

[28]  William R. King,et al.  A meta-analysis of the technology acceptance model , 2006, Inf. Manag..

[29]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[30]  Anol Bhattacherjee,et al.  Understanding Changes in Belief and Attitude Toward Information Technology Usage: A Theoretical Model and Longitudinal Test , 2004, MIS Q..

[31]  Detmar W. Straub,et al.  Validation in Information Systems Research: A State-of-the-Art Assessment , 2001, MIS Q..

[32]  Michael J. Gallivan,et al.  ORGANIZATIONS: A MULTILEVEL PERSPECTIVE , 2007 .

[33]  M. Lindell,et al.  Accounting for common method variance in cross-sectional research designs. , 2001, The Journal of applied psychology.

[34]  L. J. Williams,et al.  Recent Advances in Causal Modeling Methods for Organizational and Management Research , 2003 .

[35]  Huy Le,et al.  Beyond alpha: an empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs. , 2003, Psychological methods.

[36]  Arun Rai,et al.  Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis , 2002, Inf. Syst. Res..

[37]  F. Yammarino,et al.  Identifying Common Methods Variance With Data Collected From A Single Source: An Unresolved Sticky Issue , 1991 .

[38]  Joseph A. Cote,et al.  Estimating Trait, Method, and Error Variance: Generalizing across 70 Construct Validation Studies , 1987 .

[39]  Ann Majchrzak,et al.  Knowledge Collaboration Among Professionals Protecting National Security: Role of Transactive Memories in Ego-Centered Knowledge Networks , 2008, Organ. Sci..

[40]  Daniel Paul Manson.  Determinants and consequences of user groupware customization , 1998 .

[41]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[42]  Qingxiong Ma,et al.  The Technology Acceptance Model: A Meta-Analysis of Empirical Findings , 2004, J. Organ. End User Comput..

[43]  Charles Chowa,et al.  Information System Success: Individual and Organizational Determinants , 2006, Manag. Sci..

[44]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[45]  Joseph A. Cote,et al.  Lack of method variance in self-reported affect and perceptions at work: Reality or artifact? , 1989 .

[46]  Alan R. Dennis,et al.  Understanding Fit and Appropriation Effects in Group Support Systems via Meta-Analysis , 2001, MIS Q..

[47]  Detmar W. Straub,et al.  Veni, Vidi, Vici: Breaking the TAM Logjam , 2007, J. Assoc. Inf. Syst..

[48]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[49]  Detmar W. Straub,et al.  Information Technology Adoption Across Time: A Cross-Sectional Comparison of Pre-Adoption and Post-Adoption Beliefs , 1999, MIS Q..

[50]  Paul E. Spector.  Method variance as an artifact in self-reported affect and perceptions at work: Myth or significant problem? , 1987 .

[51]  A. Burton-Jones.  Minimizing Method Bias Through Programmatic Research , 2009 .

[52]  R. Iman,et al.  Rank Transformations as a Bridge between Parametric and Nonparametric Statistics , 1981 .

[53]  Albert L. Lederer,et al.  A Meta-Analysis of the Role of Environment-Based Voluntariness in Information Technology Acceptance , 2009, MIS Q..

[54]  Injai Kim.  The effects of individual, managerial, organizational, and environmental factors on the adoption of object orientation in United States organizations: An empirical test of the Technology Acceptance Model , 1996 .

[55]  J. Wagner,et al.  Percept-Percept Inflation in Microorganizational Research: An Investigation of Prevalence and Effect , 1994 .

[56]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[57]  Robert G. Fichman,et al.  International Conference on Information Systems ( ICIS ) 1992 INFORMATION TECHNOLOGY DIFFUSION : A REVIEW OF EMPIRICAL RESEARCH , 2017 .

[58]  John E. Hunter,et al.  Methods of Meta-Analysis , 1989 .

[59]  Naresh K. Malhotra,et al.  Common Method Variance in IS Research: A Comparison of Alternative Approaches and a Reanalysis of Past Research , 2006, Manag. Sci..

[60]  Naresh K. Malhotra,et al.  Predicting System Usage from Intention and Past Use: Scale Issues in the Predictors , 2005, Decis. Sci..

[61]  Rajeev Sharma,et al.  The Contingent Effects of Management Support and Task Interdependence on Successful Information Systems Implementation , 2003, MIS Q..

[62]  Emin Babakus,et al.  The Sensitivity of Confirmatory Maximum Likelihood Factor Analysis to Violations of Measurement Scale and Distributional Assumptions , 1987 .

[63]  WetzelsMartin,et al.  A meta-analysis of the technology acceptance model , 2007 .

[64]  Joseph A. Cote,et al.  Estimating Trait, Method, and Error Variance: Generalizing across 70 Construct Validation Studies: , 1987 .

[65]  L. J. Williams,et al.  Recent Advances in Causal Modeling Methods for Organizational and Management Research , 2003 .

[66]  Rajiv Kohli,et al.  Measuring Information Technology Payoff: A Meta - Analysis of Structural Variables in Firm - Level Empirical Research , 2003, Inf. Syst. Res..

[67]  Dan J. Putka,et al.  The Multifaceted Nature of Measurement Artifacts and Its Implications for Estimating Construct-Level Relationships , 2009 .

[68]  Detmar W. Straub,et al.  Validation Guidelines for IS Positivist Research , 2004, Commun. Assoc. Inf. Syst..

[69]  Paul A. Pavlou,et al.  Understanding and Mitigating Uncertainty in Online Exchange Relationships: A Principal-Agent Perspective , 2007, MIS Q..

[70]  Lee J. Cronbach,et al.  Giving method variance its due. , 1995 .

[71]  Detmar W. Straub,et al.  Measuring System Usage: Implications for IS Theory Testing , 1995 .