The Perils and Promises of Big Data Research in Information Systems

With the proliferation of “big data” and powerful analytical techniques, information systems (IS) researchers are increasingly engaged in what we label as big data research (BDR)—research based on large digital trace datasets and computationally intensive methods. The number of such research papers has been growing rapidly in the top IS journals during the last decade, with roughly 16% of papers in 2018 employing this approach. In this editorial, we propose five conjectures that articulate the potential consequences of increasing BDR prevalence for the IS field’s research goals and outputs. By analyzing 82 articles (41 BDR articles and a comparison group of 41 non-BDR articles) published in three of the field’s top research journals (MIS Quarterly, Information Systems Research, and Journal of Management Information Systems) during 2016-2018, we show that while the conjectures might appear controversial, they do indeed have merit. Our evidence indicates that compared to non-BDR studies, BDR studies deal less with theory and more with tactical topics. We discuss ways in which IS researchers may be able to better leverage big data and new analysis techniques to conduct more impactful research. Our intent with these conjectures and analyses is to stimulate debate in the IS community. Indeed, we need a productive discussion about how emerging new research methods, digital trace data, and the development of indigenous theory relate to and can support one another.

[1]  Brent Kitchens,et al.  Advanced Customer Analytics: Strategic Value Through Integration of Relationship-Oriented Big Data , 2018, J. Manag. Inf. Syst..

[2]  Andrew B. Whinston,et al.  Learning from Your Friends’ Check-Ins: An Empirical Study of Location-Based Social Networks , 2017, Inf. Syst. Res..

[3]  K. Sudhir Editorial - The Exploration-Exploitation Tradeoff and Efficiency in Knowledge Production , 2016, Mark. Sci..

[4]  Huimin Zhao,et al.  The Interaction Between Microblog Sentiment and Stock Returns: An Empirical Examination , 2018, MIS Q..

[5]  Roger H. L. Chiang,et al.  Big Data Research in Information Systems: Toward an Inclusive Research Agenda , 2016, J. Assoc. Inf. Syst..

[6]  A. Pentland,et al.  Life in the network: The coming age of computational social science: Science , 2009 .

[7]  Esther Gal-Or,et al.  Reward-Based Crowdfunding Campaigns: Informational Value and Access to Venture Capital , 2018, Inf. Syst. Res..

[8]  Nicholas Berente,et al.  Research Commentary - Data-Driven Computationally Intensive Theory Development , 2019, Inf. Syst. Res..

[9]  Armin Heinzl,et al.  Does Platform Owner's Entry Crowd Out Innovation? Evidence From Google Photos , 2016, ICIS.

[10]  Li Zhao,et al.  Sharing Knowledge in Social Q&A Sites: The Unintended Consequences of Extrinsic Motivation , 2016, J. Manag. Inf. Syst..

[11]  Jason D. Shaw,et al.  New Ways of Seeing Big Data , 2019, Academy of Management Journal.

[12]  Chuan-Hoo Tan,et al.  Sequentiality of Product Review Information Provision: An Information Foraging Perspective , 2017, MIS Q..

[13]  Yili Hong,et al.  Role of Monetary Incentives in the Digital and Physical Inter-Border Labor Flows , 2018, J. Manag. Inf. Syst..

[14]  Matthew J. Hashim,et al.  Information Feedback, Targeting, and Coordination: An Experimental Study , 2016, Inf. Syst. Res..

[15]  Wonseok Oh,et al.  Excessive Dependence on Mobile Social Apps: A Rational Addiction Perspective , 2016, Inf. Syst. Res..

[16]  Rikard Lindgren,et al.  Digital Traces of Information Systems: Sociomateriality made Researchable , 2013, ICIS.

[17]  Varun Grover,et al.  NEW STATE OF PLAY IN INFORMATION SYSTEMS RESEARCH : THE PUSH TO THE EDGES 1 , 2015 .

[18]  Prabuddha De,et al.  Learning Effects of Domain, Technology, and Customer Knowledge in Information Systems Development: An Empirical Study , 2017, Inf. Syst. Res..

[19]  Maurice Landry,et al.  Can the field of MIS be disciplined? , 1989, CACM.

[20]  Beibei Li,et al.  Examining the Impact of Keyword Ambiguity on Search Advertising Performance: A Topic Model Approach , 2017, MIS Q..

[21]  Jay F. Nunamaker,et al.  Exploring Emerging Hacker Assets and Key Hackers for Proactive Cyber Threat Intelligence , 2017, J. Manag. Inf. Syst..

[22]  Shane M. Greenstein,et al.  Open Content, Linus' Law, and Neutral Point of View , 2016, Inf. Syst. Res..

[23]  Naveen Kumar,et al.  Detecting Review Manipulation on Online Platforms with Hierarchical Supervised Learning , 2018, J. Manag. Inf. Syst..

[24]  Andrew M. Hardin,et al.  Assessing the Credibility of Decisional Guidance Delivered by Information Systems , 2017, J. Manag. Inf. Syst..

[25]  Alán Dennis An Unhealthy Obsession with Theory , 2019, J. Assoc. Inf. Syst..

[26]  Anindya Ghose,et al.  Towards a Digital Attribution Model: Measuring the Impact of Display Advertising on Online Consumer Behavior , 2015 .

[27]  Sandeep Purao,et al.  Data-Driven Meets Theory-Driven Research in the Era of Big Data: Opportunities and Challenges for Information Systems Research , 2018, J. Assoc. Inf. Syst..

[28]  Kai H. Lim,et al.  Nudging Moods to Induce Unplanned Purchases in Imperfect Mobile Personalization Contexts , 2018, MIS Q..

[29]  Ashish Agarwal,et al.  The Impact of Competing Ads on Click Performance in Sponsored Search , 2016, Inf. Syst. Res..

[30]  Shengli Li,et al.  A Study of Enterprise Software Licensing Models , 2017, J. Manag. Inf. Syst..

[31]  George Valkanas,et al.  The Impact of Fake Reviews on Online Visibility: A Vulnerability Assessment of the Hotel Industry , 2016, Inf. Syst. Res..

[32]  Oded Nov,et al.  Turbulent Stability of Emergent Roles: The Dualistic Nature of Self-Organizing Knowledge Coproduction , 2016, Inf. Syst. Res..

[33]  John Campbell,et al.  Temporal Motivations of Volunteers to Participate in Cultural Crowdsourcing Work , 2017, Inf. Syst. Res..

[34]  Wanda J. Orlikowski,et al.  Research Commentary: Desperately Seeking the "IT" in IT Research - A Call to Theorizing the IT Artifact , 2001, Inf. Syst. Res..

[35]  Weiguo Fan,et al.  Measuring Customer Agility from Online Reviews Using Big Data Text Analytics , 2018, J. Manag. Inf. Syst..

[36]  Rema Padman,et al.  Quantifying the Impact of Social Influence on the Information Technology Implementation Process by Physicians: A Hierarchical Bayesian Learning Approach , 2018, Inf. Syst. Res..

[37]  Michael R. Wade,et al.  Impediments to Information Systems Replacement: A Calculus of Discontinuance , 2017, J. Manag. Inf. Syst..

[38]  Anindya Ghose,et al.  Secret Admirers: An Empirical Examination of Information Hiding and Contribution Dynamics in Online Crowdfunding , 2016, Inf. Syst. Res..

[39]  Peng Huang,et al.  Participation in Open Knowledge Communities and Job-Hopping: Evidence from Enterprise Software , 2016, MIS Q..

[40]  Charles Steinfield,et al.  Strategic Visibility in Enterprise Social Media: Implications for Network Formation and Boundary Spanning , 2018, J. Manag. Inf. Syst..

[41]  Andrew B. Whinston,et al.  A Friend Like Me: Modeling Network Formation in a Location-Based Social Network , 2016, J. Manag. Inf. Syst..

[42]  David Zimbra,et al.  Targeted Twitter Sentiment Analysis for Brands Using Supervised Feature Engineering and the Dynamic Architecture for Artificial Neural Networks , 2016, J. Manag. Inf. Syst..

[43]  Zhiqiang Zheng,et al.  A Data Envelopment Analysis Approach to Estimate IT-Enabled Production Capability , 2017, MIS Q..

[44]  Wonseok Oh,et al.  Mobile App Analytics: A Multiple Discrete-Continuous Choice Framework , 2016, MIS Q..

[45]  Arun Rai,et al.  Editor's Comments: Seeing the Forest for the Trees , 2017 .

[46]  Dorothy E. Leidner,et al.  What's in a Contribution? , 2020, J. Assoc. Inf. Syst..

[47]  Wenqi Zhou,et al.  Do Professional Reviews Affect Online User Choices Through User Reviews? An Empirical Study , 2016, J. Manag. Inf. Syst..

[48]  Rudy Hirschheim,et al.  Against Theory: With Apologies to Feyerabend , 2019, J. Assoc. Inf. Syst..

[49]  David P. Biros,et al.  An Empirical Validation of Malicious Insider Characteristics , 2016, J. Manag. Inf. Syst..

[50]  Hani Safadi,et al.  Social Media Affordances for Connective Action: An Examination of Microblogging Use During the Gulf of Mexico Oil Spill , 2017, MIS Q..

[51]  Paul Jen-Hwa Hu,et al.  Top Persuader Prediction for Social Networks , 2016, MIS Q..

[52]  Varun Grover,et al.  A Historical Observation of the Intellectual and Institutional Structures of the Field , 2016, Commun. Assoc. Inf. Syst..

[53]  Marius F. Niculescu,et al.  The Double-Edged Sword of Backward Compatibility: The Adoption of Multigenerational Platforms in the Presence of Intergenerational Services , 2016, Inf. Syst. Res..

[54]  Jörg Becker,et al.  Comprehensible Predictive Models for Business Processes , 2016, MIS Q..

[55]  Fred M. Feinberg,et al.  Wearout or Weariness? Measuring Potential Negative Consequences of Online Ad Volume and Placement on Website Visits , 2019, Journal of Marketing Research.

[56]  Luis Aguiar,et al.  Catch Me if You Can: Effectiveness and Consequences of Online Copyright Enforcement , 2015, Inf. Syst. Res..

[57]  Pengzhu Zhang,et al.  Text Analytics to Support Sense-Making in Social Media: A Language-Action Perspective , 2018, MIS Q..

[58]  Robert J. Kauffman,et al.  Content Sampling, Household Informedness, and the Consumption of Digital Information Goods , 2018, J. Manag. Inf. Syst..

[59]  Jie Zhang,et al.  Expert Blogs and Consumer Perceptions of Competing Brands , 2017, MIS Q..

[60]  Jonathan Whitaker,et al.  Information Technology, Revenues and Profits: Exploring the Role of Foreign and Domestic Operations , 2016, Inf. Syst. Res..

[61]  K. Weick,et al.  Unobtrusive Measures in Organizational Theory: A Reminder. , 1979 .

[62]  Daniel Robey,et al.  Identity, Legitimacy and the Dominant Research Paradigm: An Alternative Prescription for the IS Discipline: A Response to Benbasat and Zmud's Call for Returning to the IT Artifact , 2003, J. Assoc. Inf. Syst..

[63]  Wonseok Oh,et al.  Platform Structures, Homing Preferences, and Homophilous Propensities in Online Social Networks , 2017, J. Manag. Inf. Syst..

[64]  Chong Wang,et al.  Comparing Open and Sealed Bid Auctions: Evidence from Online Labor Markets , 2013, Inf. Syst. Res..

[65]  Milind Dawande,et al.  Not Just a Fad: Optimal Sequencing in Mobile In-App Advertising , 2017, Inf. Syst. Res..

[66]  Alexander Tuzhilin,et al.  Recommending Learning Materials to Students by Identifying their Knowledge Gaps , 2014, RecSys Posters.

[67]  Jacek M. Zurada,et al.  A Data-Mining Approach to Identification of Risk Factors in Safety Management Systems , 2017, J. Manag. Inf. Syst..

[68]  J. Wu,et al.  Disconfirmation Effect on Online Rating Behavior: A Structural Model , 2014, Inf. Syst. Res..

[69]  Ravi Bapna,et al.  Monetizing Freemium Communities: Does Paying for Premium Increase Social Engagement? , 2016, MIS Q..

[70]  Xitong Li,et al.  Could Deal Promotion Improve Merchants’ Online Reputations? The Moderating Role of Prior Reviews , 2016, J. Manag. Inf. Syst..

[71]  D. J. Wu,et al.  Strategic Intellectual Property Sharing: Competition on an Open Technology Platform Under Network Effects , 2018, Inf. Syst. Res..

[72]  Wai Fong Boh,et al.  A Temporal Study of the Effects of Online Opinions: Information Sources Matter , 2017, J. Manag. Inf. Syst..

[73]  Jennifer Jie Xu,et al.  Cheap Talk? The Impact of Lender-Borrower Communication on Peer-to-Peer Lending Outcomes , 2018, J. Manag. Inf. Syst..

[74]  Shoshana Zuboff,et al.  Big other: surveillance capitalism and the prospects of an information civilization , 2015, J. Inf. Technol..

[75]  Bin Zhang,et al.  Examining Hacker Participation Length in Cybercriminal Internet-Relay-Chat Communities , 2016, J. Manag. Inf. Syst..

[76]  Izak Benbasat,et al.  The Identity Crisis Within the IS Discipline: Defining and Communicating the Discipline's Core Properties , 2003, MIS Q..

[77]  Min-Seok Pang Politics and Information Technology Investments in the U.S. Federal Government in 2003-2016 , 2017, Inf. Syst. Res..

[78]  Carsten S. Østerlund,et al.  Building an Apparatus: Refractive, Reflective, and Diffractive Readings of Trace Data , 2020, J. Assoc. Inf. Syst..

[79]  Maha Shaikh,et al.  Folding and Unfolding: Balancing Openness and Transparency in Open Source Communities , 2016, Inf. Syst. Res..

[80]  Ryan T. Wright,et al.  Research Note - Using Expectation Disconfirmation Theory and Polynomial Modeling to Understand Trust in Technology , 2016, Inf. Syst. Res..

[81]  Edoardo M. Airoldi,et al.  Assessing the Impact of Granular Privacy Controls on Content Sharing and Disclosure on Facebook , 2016, Inf. Syst. Res..

[82]  Abraham Seidmann,et al.  When Social Media Delivers Customer Service: Differential Customer Treatment in the Airline Industry , 2018, MIS Q..

[83]  Abraham Seidmann,et al.  Whose and What Social Media Complaints Have Happier Resolutions? Evidence from Twitter , 2017, J. Manag. Inf. Syst..

[84]  Elena Karahanna,et al.  The Compensatory Interaction Between User Capabiities and Technology Capabilities in Influencing Task Performance: An Empirical Assessment in Telemedicine Consultations , 2016, MIS Q..

[85]  Xitong Li,et al.  Impact of Average Rating on Social Media Endorsement: The Moderating Role of Rating Dispersion and Discount Threshold , 2017, Inf. Syst. Res..

[86]  Xia Zhao,et al.  Managerial Incentives and IT Strategic Posture , 2017, Inf. Syst. Res..

[87]  K. Lyytinen,et al.  Exploring the intellectual structures of information systems development: A social action theoretic analysis , 1996 .

[88]  Gerald C. Kane,et al.  Research Note - Content and Collaboration: An Affiliation Network Approach to Information Quality in Online Peer Production Communities , 2016, Inf. Syst. Res..

[89]  Qing Bai,et al.  How Does Social Media Impact Bitcoin Value? A Test of the Silent Majority Hypothesis , 2018, J. Manag. Inf. Syst..

[90]  Xiao Xiao,et al.  The Sociotechnical Axis of Cohesion for the IS Discipline: Its Historical Legacy and its Continued Relevance , 2019, MIS Q..

[91]  Mark Keil,et al.  Risks and Controls in Internet-Enabled Reverse Auctions: Perspectives from Buyers and Suppliers , 2017, J. Manag. Inf. Syst..

[92]  Hua Ye,et al.  User Service Innovation on Mobile Phone Platforms: Investigating Impacts of Lead Userness, Toolkit Support, and Design Autonomy , 2018, MIS Q..

[93]  Kevin Crowston,et al.  Validity Issues in the Use of Social Network Analysis with Digital Trace Data , 2011, J. Assoc. Inf. Syst..

[94]  Yili Hong,et al.  On Buyer Selection of Service Providers in Online Outsourcing Platforms for IT Services , 2017, Inf. Syst. Res..

[95]  Bin Gu,et al.  Predicting and Deterring Default with Social Media Information in Peer-to-Peer Lending , 2017, J. Manag. Inf. Syst..

[96]  Edward G. Anderson,et al.  Managing Distributed Product Development Projects: Integration Strategies for Time Zone and Language Barriers , 2017, Inf. Syst. Res..

[97]  Hind Benbya,et al.  Differential Innovativeness Outcomes of User and Employee Participation in an Online User Innovation Community , 2018, J. Manag. Inf. Syst..

[98]  Suprateek Sarker,et al.  Revisiting IS research practice in the era of big data , 2019, Inf. Organ..

[99]  Param Vir Singh,et al.  Copycats vs. Original Mobile Apps: A Machine Learning Copycat-Detection Method and Empirical Analysis , 2018, Inf. Syst. Res..

[100]  Vasant Dhar,et al.  Data science and prediction , 2012, CACM.

[101]  Vasant Dhar,et al.  Editorial - Big Data, Data Science, and Analytics: The Opportunity and Challenge for IS Research , 2014, Inf. Syst. Res..

[102]  Richard E. Boyatzis,et al.  Transforming Qualitative Information: Thematic Analysis and Code Development , 1998 .

[103]  Aron Lindberg,et al.  Developing Theory Through Integrating Human and Machine Pattern Recognition , 2020, J. Assoc. Inf. Syst..

[104]  Yili Hong,et al.  Surviving in Global Online Labor Markets for IT Services: A Geo-Economic Analysis , 2017, Inf. Syst. Res..

[105]  Panagiotis Adamopoulos,et al.  The Impact of User Personality Traits on Word of Mouth: Text-Mining Social Media Platforms , 2018, Inf. Syst. Res..

[106]  Oliver Hinz,et al.  The Impact of Sharing Mechanism Design on Content Sharing in Online Social Networks , 2018, Inf. Syst. Res..

[107]  Yili Hong,et al.  The Role of Provision Points in Online Crowdfunding , 2018, J. Manag. Inf. Syst..

[108]  Alexander Benlian,et al.  Effects of Social Interaction Dynamics on Platforms , 2016, J. Manag. Inf. Syst..

[109]  Nicholas Berente,et al.  Coordinating Interdependencies in Online Communities: A Study of an Open Source Software Project , 2016, Inf. Syst. Res..

[110]  Amit Bhatnagar,et al.  Providing a Window of Opportunity for Converting eStore Visitors , 2017, Inf. Syst. Res..

[111]  Stephan Ludwig,et al.  Untangling a Web of Lies: Exploring Automated Detection of Deception in Computer-Mediated Communication , 2016, J. Manag. Inf. Syst..

[112]  Thomas L. Huber,et al.  Governance Practices in Platform Ecosystems: Navigating Tensions Between Cocreated Value and Governance Costs , 2017, Inf. Syst. Res..

[113]  Kalle Lyytinen,et al.  Nothing At The Center?: Academic Legitimacy in the Information Systems Field , 2004, J. Assoc. Inf. Syst..