Quantitative approaches to content analysis: identifying conceptual drift across publication outlets

Unstructured text data, such as emails, blogs, contracts, academic publications, organizational documents, transcribed interviews, and even tweets, are important sources of data in Information Systems research. Various forms of qualitative analysis of the content of these data exist and have revealed important insights. Yet, to date, these analyses have been hampered by limitations of human coding of large data sets, and by bias due to human interpretation. In this paper, we compare and combine two quantitative analysis techniques to demonstrate the capabilities of computational analysis for content analysis of unstructured text. Specifically, we seek to demonstrate how two quantitative analytic methods, viz., Latent Semantic Analysis and data mining, can aid researchers in revealing core content topic areas in large (or small) data sets, and in visualizing how these concepts evolve, migrate, converge or diverge over time. We exemplify the complementary application of these techniques through an examination of a 25-year sample of abstracts from selected journals in Information Systems, Management, and Accounting disciplines. Through this work, we explore the capabilities of two computational techniques, and show how these techniques can be used to gather insights from a large corpus of unstructured text.

[1]  Charles-Clemens Rüling,et al.  Popular concepts and the business management press , 2005 .

[2]  Sorrek Penn-Edwards,et al.  Computer Aided Phenomenography: The R ole of Leximancer Computer Soft ware in Phenomenographic Investigation , 2010 .

[3]  Olivier Poch,et al.  Knowledge-based expert systems and a proof-of-concept case study for multiple sequence alignment construction and analysis , 2009, Briefings Bioinform..

[4]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[5]  Marta Indulska,et al.  Design science in IS research : a literature analysis , 2008 .

[6]  Kar Yan Tam,et al.  The Impact of Information Technology Investments on Firm Performance and Evaluation: Evidence from Newly Industrialized Economies , 1998, Inf. Syst. Res..

[7]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[8]  Daniel G. Bobrow,et al.  Community Knowledge Sharing in Practice: The Eureka Story , 2002 .

[9]  K. Seers Qualitative data analysis , 2011, Evidence Based Nursing.

[10]  Peter Wiemer-Hastings,et al.  Latent semantic analysis , 2004, Annu. Rev. Inf. Sci. Technol..

[11]  Rosann Webb Collins,et al.  Technology Requirements and Work Group Communication for Telecommuters , 2001, Inf. Syst. Res..

[12]  Richard Baskerville,et al.  Fashion Waves in Information Systems Research and Practice , 2009, MIS Q..

[13]  David Yarowsky,et al.  Unsupervised Word Sense Disambiguation Rivaling Supervised Methods , 1995, ACL.

[14]  Cathy Urquhart,et al.  Putting the ‘theory’ back into grounded theory: guidelines for grounded theory studies in information systems , 2009, Inf. Syst. J..

[15]  Patrick F. Reidy An Introduction to Latent Semantic Analysis , 2009 .

[16]  Sirkka L. Jarvenpaa,et al.  An Information Company in Mexico: Extending the Resource-Based View of the Firm to a Developing Country Context , 1998, Inf. Syst. Res..

[17]  G. Northcraft,et al.  Serving Constituencies In Business Schools: M.B.A. Program Versus Research Performance , 2000 .

[18]  Andy Dong,et al.  The latent semantic approach to studying design team communication , 2005 .

[19]  T. Landauer LSA as a Theory of Meaning , 2007 .

[20]  R. Gorsuch Exploratory factor analysis: its role in item analysis. , 1997, Journal of personality assessment.

[21]  Danielle S. McNamara,et al.  Handbook of latent semantic analysis , 2007 .

[22]  Gerard Salton,et al.  Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer , 1989 .

[23]  Dirk S. Hovorka,et al.  Analyzing unstructured text data: Using latent categorization to identify intellectual communities in information systems , 2008, Decis. Support Syst..

[24]  Shirley Gregor,et al.  Information Systems Foundations: The Role of Design Science , 2011 .

[25]  Yajiong Xue,et al.  Avoidance of Information Technology Threats: A Theoretical Perspective , 2009, MIS Q..

[26]  Victor R. Prybutok,et al.  Latent Semantic Analysis: five methodological recommendations , 2012, Eur. J. Inf. Syst..

[27]  R. Weber Basic Content Analysis , 1986 .

[28]  Ralph D. Loftin,et al.  SIM Competition Paper: Organization Development Methods in the Management of the Information Systems Function , 1982, MIS Q..

[29]  Lee L. Gremillion Managing the Implementation of Standardized Computer Based Systems , 1980, MIS Q..

[30]  Pairin Katerattanakul,et al.  Is information systems a reference discipline? , 2006, CACM.

[31]  Janet Wiles,et al.  Use of an automatic content analysis tool: A technique for seeing both local and global scope , 2009, Int. J. Hum. Comput. Stud..

[32]  T. Goldberg,et al.  Quantifying incoherence in speech: An automated methodology and novel application to schizophrenia , 2007, Schizophrenia Research.

[33]  Kai R. Larsen,et al.  9. A Mathematical Approach to Categorization and Labeling of Qualitative Data: The Latent Categorization Method , 2004 .

[34]  M. F. Porter,et al.  An algorithm for suffix stripping , 1997 .

[35]  Ping Wang,et al.  Research Directions in Information Systems: Toward an Institutional Ecology , 2008, J. Assoc. Inf. Syst..

[36]  James C. Wetherbe,et al.  Heuristic Development: A Redesign of Systems Design , 1979, MIS Q..

[37]  Kristof Coussement,et al.  Improving Customer Complaint Management by Automatic Email Classification Using Linguistic Style Features as Predictors , 2007 .

[38]  Bernard McKenna,et al.  Media-ted political oratory following terrorist events: International political responses to the 2005 London bombing , 2007 .

[39]  Anna Sidorova,et al.  Uncovering the Intellectual Core of the Information Systems Discipline , 2008, MIS Q..

[40]  Andrew E. Smith,et al.  Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping , 2006, Behavior research methods.

[41]  Chih-Ping Wei,et al.  A Latent Semantic Indexing-based approach to multilingual document clustering , 2008, Decis. Support Syst..

[42]  Walter Fernandez,et al.  Using the Glaserian Approach in Grounded Studies of Emerging Business Practices , 2004 .

[43]  Venkataraman Ramesh,et al.  Research in Information Systems: An Empirical Study of Diversity in the Discipline and Its Journals , 2002, J. Manag. Inf. Syst..

[44]  Zainab Monjed AlQenaei An investigation of the relationship between consumer mental health recovery indicators and clinicians' reports using multivariate analyses of the singular value decomposition of a textual corpus , 2009 .

[45]  Wei-Pang Yang,et al.  Text summarization using a trainable summarizer and latent semantic analysis , 2005, Inf. Process. Manag..

[46]  Jerome Kanter,et al.  Developing an expert systems strategy , 1989 .

[47]  S. Dumais Latent Semantic Analysis. , 2005 .

[48]  James A. Hampton,et al.  Testing the Prototype Theory of Concepts , 1995 .

[49]  Marta Indulska,et al.  How do practitioners use conceptual modeling in practice? , 2006, Data Knowl. Eng..

[50]  John P. Rice,et al.  Profiling Enterprise Risks in Large Computer Companies Using the Leximancer Software Tool , 2007 .

[51]  Curt Burgess,et al.  Modelling Parsing Constraints with High-dimensional Context Space , 1997 .

[52]  William R. King Text Analytics: Boon to Knowledge Management? , 2009, Inf. Syst. Manag..

[53]  Dirk S. Hovorka,et al.  Conceptual convergences: Positioning information systems among the business disciplines , 2009, ECIS.

[54]  C. Lawrence Meador,et al.  Setting Priorities for DSS Development , 1984, MIS Q..

[55]  Peter W. Foltz Improving human-proceedings interaction: indexing the CHI index , 1995, CHI '95.

[56]  Ana Ortiz de Guinea,et al.  Why break the habit of a lifetime? rethinking the roles of intention, habit, and emotion in continuing information technology use , 2009 .

[57]  William M. Pottenger,et al.  A framework for understanding Latent Semantic Indexing (LSI) performance , 2006, Inf. Process. Manag..

[58]  S. Paxton,et al.  Pathways to help-seeking in bulimia nervosa and binge eating problems: a concept mapping approach. , 2007, The International journal of eating disorders.

[59]  Yi Zhao,et al.  Antecedents of the Closeness of Human-Avatar Relationships in a Virtual World , 2010, J. Database Manag..

[60]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[61]  Carol Saunders,et al.  Management Information Systems, Communications, and Departmental Power: An Integrative Model , 1981 .

[62]  A. Dennis,et al.  Serving Multiple Constituencies in the Business School: MBA Program vs. Research Performance , 2000 .