Understanding Relationship between Scholars' Breadth of Research and Scientific Impact

Many existing metrics to evaluate scholars consider their scientific impact without considering the importance of breadth of research. In this paper, we define a new metric for breadth of research based on the generalized Stirling metric that considers multiple aspects of breadth of research. We extract research topics in computer science using concept extraction and clustering from the literature in the ACM dataset. We then assign authors a distribution over these research topics, from which we calculate scores of breadth of research for each author. We design five simulation experiments that evaluate the ability of a metric to measure breadth of research and use these experiments to compare our new metric to traditional metrics. The results show how these metrics perform in different experiments, concluding that no metric consistently outperforms the others. We test the relationship between our new metric and scientific impact and find a weak correlation between them. Finally, we find that the variation of the metric over time illustrates a possible publication pattern for scholars. Conference Topic Indicators

[1]  T. D. N. Thi,et al.  Why do academic scientists engage in interdisciplinary research , 2005 .

[2]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[3]  Ismael Rafols,et al.  How journal rankings can suppress interdisciplinary research: A comparison between Innovation Stud , 2012 .

[4]  Alan L. Porter,et al.  Validating indicators of interdisciplinarity: linking bibliometric measures to studies of engineering research labs , 2012, Scientometrics.

[5]  Lorenzo Cassi,et al.  How to evaluate the degree of interdisciplinarity of an institution? , 2014, Scientometrics.

[6]  Christian Sternitzke,et al.  Similarity measures for document mapping: A comparative study on the level of an individual scientist , 2007, Scientometrics.

[7]  Marie Hastings-Tolsma,et al.  Measurement of Diversity of Human Field Pattern , 2011 .

[8]  Ismael Rafols,et al.  Diversity and network coherence as indicators of interdisciplinarity: case studies in bionanoscience , 2009, Scientometrics.

[9]  E. H. Simpson Measurement of Diversity , 1949, Nature.

[10]  Carl Lagoze,et al.  The extraction of community structures from publication networks to support ethnographic observations of field differences in scientific communication , 2012, J. Assoc. Inf. Sci. Technol..

[11]  Bing He,et al.  Community-based topic modeling for social tagging , 2010, CIKM.

[12]  J. Ruscio,et al.  Measuring Scholarly Impact Using Modern Citation-Based Indices , 2012 .

[13]  Martin Rosvall,et al.  Maps of random walks on complex networks reveal community structure , 2007, Proceedings of the National Academy of Sciences.

[14]  Kevin W. Boyack,et al.  Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature , 2011, J. Informetrics.

[15]  Dunja Mladenic,et al.  Interdisciplinarity of scientific fields and its evolution based on graph of project collaboration and co-authoring , 2014, Scientometrics.

[16]  Filippo Menczer,et al.  Universality of scholarly impact metrics , 2013, J. Informetrics.

[17]  Erjia Yan,et al.  Finding knowledge paths among scientific disciplines , 2013, J. Assoc. Inf. Sci. Technol..

[18]  Claude E. Shannon,et al.  Recent Contributions to The Mathematical Theory of Communication , 2009 .

[19]  Luciano da Fontoura Costa,et al.  Quantifying the interdisciplinarity of scientific journals and fields , 2012, J. Informetrics.

[20]  F. J. Rijnsoever,et al.  Factors associated with disciplinary and interdisciplinary research collaboration , 2011 .

[21]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[22]  Alan L. Porter,et al.  Measuring researcher interdisciplinarity , 2007, Scientometrics.

[23]  A. Stirling A general framework for analysing diversity in science, technology and society , 2007, Journal of The Royal Society Interface.

[24]  Mike Thelwall,et al.  Is multidisciplinary research more highly cited? A macrolevel study , 2008 .

[25]  Carl Lagoze,et al.  The web of topics: discovering the topology of topic evolution in a corpus , 2011, WWW.

[26]  Peter Weingart,et al.  Impact of bibliometrics upon the science system: Inadvertent consequences? , 2005, Scientometrics.

[27]  Duane E. Williams,et al.  Breakthrough paper indicator 2.0: can geographical diversity and interdisciplinarity improve the accuracy of outstanding papers prediction? , 2014, Scientometrics.

[28]  Pablo Jensen,et al.  The many dimensions of laboratories’ interdisciplinarity , 2013, Scientometrics.

[29]  Leah G. Nichols A topic model approach to measuring interdisciplinarity at the National Science Foundation , 2014, Scientometrics.

[30]  Bing He,et al.  Mining diversity subgraph in multidisciplinary scientific collaboration networks: A meso perspective , 2013, J. Informetrics.

[31]  Johan Bollen,et al.  A Principal Component Analysis of 39 Scientific Impact Measures , 2009, PloS one.