Unraveling Scientific Impact

textabstractThe number of citations a paper receives is the most commonly used measure of scientific impact. In this paper, we study not only the number but also the type of citations that 659 marketing articles generated. We discern five citation types: application, affirmation, negation, review and perfunctory mention (i.e., citing an article only indirectly without really using it). Prior literature in scientometrics recognizes that the former three types, on average, signal a higher level of scientific indebtedness than the latter two types. In our sample, these three types of citation represent only 15% of all citations. We also find different determinants of citation behavior across citation types. Across the 49 determinants we included, only 13 have the same effect across all citation types, of which only 5 are statistically significant across all citation types. For instance, we find a significant inverted U-effect of challenging commonly held beliefs on citations counts, but only for three of the citation types: affirmation, review and perfunctory mention. Our results encourage scientific stakeholders to move beyond mere citation counts to assess a paper’s or a scholar’s scientific contribution, as well as to devote greater attention to the citation process itself.

[1]  Murray S. Davis,et al.  That's Interesting! , 1971 .

[2]  A. Cameron,et al.  Regression-based tests for overdispersion in the Poisson model☆ , 1990 .

[3]  D. Cases,et al.  How can we investigate citation behavior?: a study of reasons for citing literature in communication , 2000 .

[4]  J. Hirsch.  Does the h index have predictive power? , 2007, Proceedings of the National Academy of Sciences.

[5]  Mandeep Singh,et al.  Marketing Renaissance: Opportunities and Imperatives for Improving Marketing Thought, Practice, and Infrastructure , 2005 .

[6]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[7]  R. Pieters,et al.  The Structural Influence of Marketing Journals: A Citation Analysis of the Discipline and its Subareas over Time , 2003 .

[8]  C. Moorman,et al.  Cross-Sectional versus Longitudinal Survey Research: Concepts, Findings, and Guidelines , 2008 .

[9]  S. Rynes,et al.  What Makes Management Research Interesting, And Why Does It Matter? , 2006 .

[10]  Douglas Walton,et al.  Appeal to Expert Opinion: Arguments from Authority , 1997 .

[11]  Lutz Bornmann,et al.  What do citation counts measure? A review of studies on citing behavior , 2008, J. Documentation.

[12]  Bernard Barber,et al.  Resistance by scientists to scientific discovery. , 1961, Science.

[13]  FROM THE EDITORS PUBLISHING IN AMJ — PART 5 : CRAFTING THE METHODS AND RESULTS , 2012 .

[14]  Hao Wang.  Do Returns Policies Intensify Retail Competition , 2004 .

[15]  Daniel C. Smith.  The importance and challenges of being interesting , 2003 .

[16]  Terrence A. Brooks,et al.  Private acts and public objects: An investigation of citer motivations , 1985, J. Am. Soc. Inf. Sci..

[17]  B BARBER,et al.  Resistance by Scientists to Scientific Discovery , 1963 .

[18]  Joel Huber.  The Value of Sticky Articles , 2008 .

[19]  Günter Krampen,et al.  On the validity of citation counting in science evaluation: Content analyses of references and citations in psychological publications , 2007, Scientometrics.

[20]  Peter Vinkler,et al.  A quasi-quantitative citation model , 1987, Scientometrics.

[21]  Claudio Castellano,et al.  Universality of citation distributions: Toward an objective measure of scientific impact , 2008, Proceedings of the National Academy of Sciences.

[22]  N. Malhotra,et al.  Marketing Research: An Applied Approach , 2000 .

[23]  T. Judge,et al.  What Causes a Management Article to be Cited—Article, Author, or Journal? , 2007 .

[24]  R. Abelson.  Statistics As Principled Argument , 1995 .

[25]  Dominique M. Hanssens,et al.  Sustained Spending and Persistent Response: A New Look at Long-Term Marketing Profitability , 1999 .

[26]  S. Samiee,et al.  Currency Choice in Industrial Pricing: A Cross-National Evaluation , 1998 .

[27]  Martin Buxton,et al.  Using categorisations of citations when assessing the outcomes from health research , 2005, Scientometrics.

[28]  Mark B. Houston,et al.  The Impact of Article Method Type and Subject Area on Article Citations and Reference Diversity in JM, JMR, and JCR , 2001 .

[29]  Steve Woolgar,et al.  Beyond the citation debate: towards a sociology of measurement technologies and their use in science policy , 1991 .

[30]  M. Yadav.  The Decline of Conceptual Articles and Implications for Knowledge Development , 2010 .

[31]  M. Mahoney.  Psychology of the Scientist: An Evaluative Review , 2016 .

[32]  J. Swait,et al.  Brands as Signals: A Cross-Country Validation Study: , 2006 .

[33]  Betty J. Simkins,et al.  Journal Communication and Influence in Financial Research , 1994 .

[34]  Gerard George,et al.  Publishing in AMJ: Part 1: Topic choice: From the Editors , 2011 .

[35]  Bluma C. Peritz,et al.  A classification of citation roles for the social sciences and related fields , 1983, Scientometrics.

[36]  FROM THE EDITORS PUBLISHING IN AMJ—PART 1: TOPIC CHOICE , 2011 .

[37]  Jacob Goldenberg,et al.  Special Issue on Design and Development: The Idea Itself and the Circumstances of Its Emergence as Predictors of New Product Success , 2001, Manag. Sci..

[38]  V. Cano,et al.  Citation behavior: Classification, utility, and location , 1989, JASIS.

[39]  D. Lehmann,et al.  From the (Past) Editors , 2009 .

[40]  J. Guzmán.  Regression Models for Categorical Dependent Variables Using Stata , 2013 .

[41]  Stephen Cole,et al.  Social Stratification in Science. , 1973 .

[42]  David A. Griffith,et al.  What does it Take to Get Promoted in Marketing Academia? Understanding Exceptional Publication Productivity in the Leading Marketing Journals , 2009 .

[43]  Peter Y. T. Sun,et al.  What have scholars retrieved from Walsh and Ungson (1991)? A citation context study , 2010 .

[44]  Jacob Goldenberg,et al.  Database Submission - The Evolving Social Network of Marketing Scholars , 2010, Mark. Sci..

[45]  Donald R. Lehmann,et al.  Sophistication in Research in Marketing , 2011 .

[46]  S. Neslin,et al.  Promotional Elasticities and Category Characteristics , 1996 .

[47]  Donald Owen Case,et al.  How can we investigate citation behavior? A study of reasons for citing literature in communication , 2000, J. Am. Soc. Inf. Sci..

[48]  C. O. Frost.  The Use of Citations in Literary Research: A Preliminary Classification of Citation Functions , 1979, The Library Quarterly.

[49]  Murray Turoff,et al.  The Delphi Method: Techniques and Applications , 1976 .

[50]  R. Merton.  The Matthew Effect in Science, II: Cumulative Advantage and the Symbolism of Intellectual Property , 1988, Isis.

[51]  Daryl E. Chubin,et al.  Content Analysis of References: Adjunct or Alternative to Citation Counting? , 1975 .

[52]  E. Garfield.  How ISI selects journals for coverage: Quantitative and qualitative considerations , 1990 .

[53]  Pradeep K. Tyagi,et al.  Met Expectations and Turnover in Direct Selling , 1991 .

[54]  Kène Henkens,et al.  What Makes a Scientific article Influential , 2000 .

[55]  Fang Xu,et al.  The drivers of citations in management science journals , 2010, Eur. J. Oper. Res..

[56]  S. Baldi.  Normative versus social constructivist processes in the allocation of citations : A network-analytic model , 1998 .

[57]  M. Moravcsik,et al.  Some Results on the Function and Quality of Citations , 1975 .

[58]  E. Garfield.  Citation analysis as a tool in journal evaluation. , 1972, Science.

[59]  R. Jackson,et al.  The Matthew Effect in Science , 1988, International journal of dermatology.

[60]  Donald D. Bergh,et al.  Some predictors of SMJ article impact , 2006 .

[61]  T. Kuhn.  The Structure of Scientific Revolutions 2nd edition , 1970 .

[63]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[64]  Suzanne Hidi,et al.  Interest and Its Contribution as a Mental Resource for Learning , 1990 .

[65]  Paul J. Silvia,et al.  Curiosity and Interest: The Benefits of Thriving on Novelty and Challenge , 2009 .

[66]  John Roberts,et al.  From academic research to marketing practice: Exploring the marketing science value chain , 2014, How to Get Published in the Best Marketing Journals.

[67]  Jacob Goldenberg,et al.  The evolving social network of marketing scholars , 2010 .

[68]  Peter E. Rossi,et al.  A Bayesian Approach to Estimating Household Parameters , 1993 .

[69]  Steven M. Shugan.  Editorial: Defining Interesting Research Problems , 2003 .

[70]  Henry G. Small,et al.  On the shoulders of Robert Merton: Towards a normative theory of citation , 2004, Scientometrics.

[71]  R. Lusch,et al.  Getting Marketers to Invest in Firm-Specific Capital , 2007 .

[72]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[73]  Alan G. Sawyer,et al.  The Readability of Marketing Journals: Are Award-Winning Articles Better Written? , 2008 .

[74]  W. Kunz,et al.  Toward a Deeper Understanding of Service Marketing: The Past, the Present, and the Future , 2011 .

[75]  I. Png,et al.  Manufacturer's Return Policies and Retail Competition , 1997 .

[76]  Leonidas C. Leonidou,et al.  Marketing strategy determinants of export performance: a meta-analysis , 2002 .

[77]  A. Cameron,et al.  Count Panel Data , 2015 .

[78]  David S. Ackerman,et al.  In Search of Diversity: The Record of Major Marketing Journals , 1999 .

[79]  鄭宇庭.  行銷硏究 : Marketing research , 2009 .

[80]  T. Kuhn.  The structure of scientific revolutions, 3rd ed. , 1996 .

[81]  W. Shadish,et al.  Author Judgements about Works They Cite: Three Studies from Psychology Journals , 1995 .

[82]  P. Verhoef,et al.  Globalization of Authorship in the Marketing Discipline: Does It Help or Hinder the Field? , 2005 .

[83]  M. Dekimpe.  From the (New) editor , 2009 .

[84]  Leonard M. Lodish,et al.  An Implemented System for Improving Promotion Productivity Using Store Scanner Data , 1993 .