Eight Obstacles to Overcome in the Theory Testing Genre

Theory testing work is popular in information systems (IS), with many studies using questionnaires, experiments, or other methods to gather quantitative data and test hypotheses with statistical techniques. This editorial note highlights some of the obstacles that theory testing researchers face, and assists authors so that their papers will not be rejected outright on submission, nor slowed unnecessarily in the review process. In particular, we identify three obstacles relating to theorizing and five related to methods. We provide guidance on how authors can deal with each obstacle, and include examples of studies that have successfully addressed the obstacle. We hope that our editorial will encourage authors to take better note of these obstacles and to give further consideration to avoiding them when they plan and conduct their research studies.

[1]  Daniel C. Feldman,et al.  The Devil is in the Details: Converting Good Research into Publishable Articles , 2004 .

[2]  J. Colquitt,et al.  TRENDS IN THEORY BUILDING AND THEORY TESTING: A FIVE-DECADE STUDY OF THE ACADEMY OF MANAGEMENT JOURNAL , 2007 .

[3]  S. Donaldson,et al.  Understanding Self-Report Bias in Organizational Behavior Research , 2002 .

[4]  N. Schmitt.  Method bias: The importance of theory and measurement , 1994 .

[5]  Qing Chang,et al.  How Low Should You Go? Low Response Rates and the Validity of Inference in IS Questionnaire Research , 2006, J. Assoc. Inf. Syst..

[6]  A. Burton-Jones.  Minimizing Method Bias Through Programmatic Research , 2009 .

[7]  Jörg Henseler,et al.  Handbook of Partial Least Squares: Concepts, Methods and Applications , 2010 .

[8]  Alan R. Hevner,et al.  Towards a NeuroIS Research Methodology: Intensifying the Discussion on Methods, Tools, and Measurement , 2014, J. Assoc. Inf. Syst..

[9]  Delroy L. Paulhus,et al.  Balanced Inventory of Desirable Responding , 2012 .

[10]  Carla H. Lagorio,et al.  Psychology , 1929, Nature.

[11]  Neuman.  Social Research Methods , 2007 .

[12]  Traci Carte,et al.  In Pursuit of Moderation: Nine Common Errors and Their Solutions , 2003, MIS Q..

[13]  K. Weick.  What Theory Is Not, Theorizing Is , 1995 .

[14]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[15]  Rajeev Sharma,et al.  Estimating the effect of common method variance: the method-method pair technique with an illustration from TAM research , 2009 .

[16]  Detmar W. Straub,et al.  Validation in Information Systems Research: A State-of-the-Art Assessment , 2001, MIS Q..

[17]  Apa Publications,et al.  Reporting standards for research in psychology: why do we need them? What might they be? , 2008, The American psychologist.

[18]  John P. Geyman,et al.  Evidence-Based Medicine in Primary Care: An Overview , 1998, The Journal of the American Board of Family Medicine.

[19]  Paul A. Pavlou,et al.  Understanding and Predicting Electronic Commerce Adoption: An Extension of the Theory of Planned Behavior , 2006, MIS Q..

[20]  Izak Benbasat,et al.  The Effects of Service and Consumer Product Knowledge on Online Customer Loyalty , 2011, J. Assoc. Inf. Syst..

[21]  Richard T. Watson,et al.  Analyzing the Past to Prepare for the Future: Writing a Literature Review , 2002, MIS Q..

[22]  L. G. Pee,et al.  Knowledge Sharing in Information Systems Development: A Social Interdependence Perspective , 2010, J. Assoc. Inf. Syst..

[23]  Scott B. MacKenzie,et al.  Working memory: theories, models, and controversies. , 2012, Annual review of psychology.

[24]  Gi Mun Kim,et al.  IT Capabilities, Process-Oriented Dynamic Capabilities, and Firm Financial Performance , 2011, J. Assoc. Inf. Syst..

[25]  Tamás D. Gedeon,et al.  Neuroscience and a Nomological Network for the Understanding and Assessment of Emotions in Information Systems Research , 2014, J. Manag. Inf. Syst..

[26]  Ron Weber,et al.  Evaluating and Developing Theories in the Information Systems Discipline , 2012, J. Assoc. Inf. Syst..

[27]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[28]  Chris W. Clegg,et al.  Explaining intranet use with the technology acceptance model , 2001, J. Inf. Technol..

[29]  Varun Grover,et al.  Investigating Two Contradictory Views of Formative Measurement in Information Systems Research , 2010, MIS Q..

[30]  Marvin D. Troutt,et al.  Decision Factors for the Adoption and Continued Use of Online Direct Sales Channels among SMEs , 2011, J. Assoc. Inf. Syst..

[31]  Yajiong Xue,et al.  Understanding Security Behaviors in Personal Computer Usage: A Threat Avoidance Perspective , 2010, J. Assoc. Inf. Syst..

[32]  Peter B. Seddon,et al.  Towards the improved treatment of generalization of knowledge claims in IS research: drawing general conclusions from samples , 2012, Eur. J. Inf. Syst..

[33]  Adamantios Diamantopoulos,et al.  Advancing formative measurement models , 2008 .

[34]  Daniel C. Feldman.  What are We Talking About When We Talk About Theory? , 2004 .

[35]  D. Straub,et al.  Editor's comments: a critical look at the use of PLS-SEM in MIS quarterly , 2012 .

[36]  Jeffrey R. Edwards,et al.  The Fallacy of Formative Measurement , 2011 .

[37]  P. Bentler,et al.  Formative Constructs Implemented via Common Factors , 2011 .

[38]  C. Hempel.  Philosophy of Natural Science , 1966 .

[39]  Dirk S. Hovorka,et al.  Conceptual convergences: Positioning information systems among the business disciplines , 2009, ECIS.

[40]  Scott B. MacKenzie,et al.  Construct Measurement and Validation Procedures in MIS and Behavioral Research: Integrating New and Existing Techniques , 2011, MIS Q..

[41]  Stephen R. Barley,et al.  When I Write My Masterpiece: Thoughts On What Makes A Paper Interesting , 2006 .

[42]  Marko Sarstedt,et al.  PLS-SEM: Indeed a Silver Bullet , 2011 .

[43]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[44]  J. Rossiter,et al.  Tailor-made single-item measures of doubly concrete constructs , 2009 .

[45]  Ryan T. Wright,et al.  Assessing Common Method Bias: Problems with the ULMC Technique , 2012, MIS Q..

[46]  M. Lynne Markus,et al.  Industry-Wide Information Systems Standardization as Collective Action: The Case of the U.S. Residential Mortgage Industry , 2006, MIS Q..

[47]  Marshall Schminke.  From the Editors: Raising the Bamboo Curtain , 2004 .

[48]  D. Whetten.  What Constitutes a Theoretical Contribution , 1989 .

[49]  Gary Klein,et al.  Resolving Difference Score Issues in Information Systems Research , 2009, MIS Q..

[50]  Ray J. Paul,et al.  Editor's View: an opportunity for editors of IS journals to relate their experiences and offer advice. The Editorial View of Ray J Paul. First in a series , 2005, Eur. J. Inf. Syst..

[51]  Naresh K. Malhotra,et al.  Common Method Variance in IS Research: A Comparison of Alternative Approaches and a Reanalysis of Past Research , 2006, Manag. Sci..