Measuring the impact of novelty, bibliometric, and academic-network factors on citation count using a neural network

Abstract The factors influencing academic citations have been extensively discussed in the literature. However, few studies have investigated whether atypical recombinations of references, topics, and keywords, and academic-network factors (e.g., author citation network-related factors, co-author network-related factors, and institution citation network-related factors) are correlated with paper citation counts. Also, most previous studies have only focused on one discipline. Twenty-four factors were classified into three main categories, including the novelty of the paper, bibliometric indicators, and the influence of the academic network of authors and institutions, which have not yet been simultaneously considered. To fill this gap in the literature, a neural network model was constructed to measure the influence of these 24 factors on citation counts using the weight product of connecting neurons. The results demonstrated that the influence of novelty, bibliometric, and academic-network-related factors on citation counts vary significantly among the four studied disciplines (library & information science, nuclear science & technology, computer science & software engineering, and history). It was found that the influence of multiple factors in the novelty category on citation counts is higher than the bibliometric and academic-network categories, while the individual factor in the novelty category is not always the most influential factor (median z-score, recombination topic pairs, and recombination keyword pairs).

[1]  Loet Leydesdorff,et al.  Betweenness centrality as a driver of preferential attachment in the evolution of research collaboration networks , 2011, J. Informetrics.

[2]  Altmetrics: A new paradigm for scholarly communication , 2019, Indian journal of dental research : official publication of Indian Society for Dental Research.

[3]  John Skvoretz,et al.  A regression analysis of researchers' social network metrics on their citation performance in a college of engineering , 2014, J. Informetrics.

[4]  G. David Garson,et al.  Interpreting neural-network connection weights , 1991 .

[5]  Scott Chamberlain,et al.  Consuming Article-Level Metrics: Observations and Lessons , 2013 .

[6]  Julian D. Olden,et al.  Illuminating the “black box”: a randomization approach for understanding variable contributions in artificial neural networks , 2002 .

[7]  Melissa A. Schilling,et al.  Recombinant Search and Breakthrough Idea Generation: An Analysis of High Impact Papers in the Social Sciences , 2011 .

[8]  G. Antoniou,et al.  Bibliometric analysis of factors predicting increased citations in the vascular and endovascular literature. , 2015, Annals of vascular surgery.

[9]  Ernesto Roldan-Valadez,et al.  Alternative bibliometrics from impact factor improved the esteem of a journal in a 2-year-ahead annual-citation calculation: multivariate analysis of gastroenterology and hepatology journals , 2015, European journal of gastroenterology & hepatology.

[10]  Loet Leydesdorff,et al.  How are new citation-based journal indicators adding to the bibliometric toolbox? , 2009, J. Assoc. Inf. Sci. Technol..

[11]  Xiaomei Bai,et al.  Predicting the citations of scholarly paper , 2019, J. Informetrics.

[12]  Ruchi Mittal,et al.  Identifying Prominent Authors from Scientific Collaboration Multiplex Social Networks , 2019 .

[13]  Peter Schlattmann,et al.  Impact of Article Language in Multi-Language Medical Journals - a Bibliometric Analysis of Self-Citations and Impact Factor , 2013, PloS one.

[14]  S. Agatonovic-Kustrin,et al.  Basic concepts of artificial neural network (ANN) modeling and its application in pharmaceutical research. , 2000, Journal of pharmaceutical and biomedical analysis.

[15]  Giovanni Felici,et al.  Predicting publication long-term impact through a combination of early citations and journal impact factor , 2019, J. Informetrics.

[16]  John Rigby,et al.  Looking for the impact of peer review: does count of funding acknowledgements really predict research impact? , 2012, Scientometrics.

[17]  M. Henchion,et al.  Making sense of altmetrics: The perceived threats and opportunities for academic identity , 2019, Science and Public Policy.

[18]  Loet Leydesdorff,et al.  The Relative Influences of Government Funding and International Collaboration on Citation Impact , 2017, J. Assoc. Inf. Sci. Technol..

[19]  Sen Chai,et al.  Breakthrough recognition: Bias against novelty and competition for attention , 2019, Research Policy.

[20]  Griffin M. Weber,et al.  The Power of Research Networking Systems to Find Experts and Facilitate Collaboration , 2019 .

[21]  Jakoah Brgoch,et al.  Evaluating Thermal Quenching Temperature in Eu3+-Substituted Oxide Phosphors via Machine Learning. , 2019, ACS applied materials & interfaces.

[22]  D J PRICE,et al.  NETWORKS OF SCIENTIFIC PAPERS. , 1965, Science.

[23]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[24]  S. Carroll Brave Genius: A Scientist, a Philosopher, and Their Daring Adventures from the French Resistance to the Nobel Prize , 2013 .

[25]  Jerome K. Vanclay,et al.  Factors affecting citation rates in environmental science , 2013, J. Informetrics.

[26]  Ying Ding,et al.  Applying centrality measures to impact analysis: A coauthorship network analysis , 2009, J. Assoc. Inf. Sci. Technol..

[27]  Russell G. Death,et al.  An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data , 2004 .

[28]  John Qi Dong,et al.  Information technology and innovation outcomes: is knowledge recombination the missing link? , 2019, EJIS.

[29]  Kevin W. Boyack,et al.  Indicator-assisted evaluation and funding of research: Visualizing the influence of grants on the number and citation counts of research papers , 2003, J. Assoc. Inf. Sci. Technol..

[30]  Lutz Bornmann,et al.  What factors determine citation counts of publications in chemistry besides their quality? , 2012, J. Informetrics.

[31]  Yi Zhang,et al.  Measuring academic influence using heterogeneous author-citation networks , 2019, Scientometrics.

[32]  Min Song,et al.  Representing Scientific Knowledge: The Role of Uncertainty , 2017 .

[33]  Qing Ke,et al.  A probe into 66 factors which are possibly associated with the number of citations an article received , 2019, Scientometrics.

[34]  Jian Wang,et al.  Bias Against Novelty in Science: A Cautionary Tale for Users of Bibliometric Indicators , 2015 .

[35]  Ronald Rousseau,et al.  Social network analysis: a powerful strategy, also for the information sciences , 2002, J. Inf. Sci..

[36]  David J. Currie,et al.  Big Science vs. Little Science: How Scientific Impact Scales with Funding , 2013, PloS one.

[37]  Tian Yu,et al.  Citation impact prediction for scientific papers using stepwise regression analysis , 2014, Scientometrics.

[38]  Britta Ruhnau,et al.  Eigenvector-centrality - a node-centrality? , 2000, Soc. Networks.

[39]  Jian Wang,et al.  How to improve the prediction based on citation impact percentiles for years shortly after the publication date? , 2013, J. Informetrics.

[40]  L. Egghe,et al.  Theory and practise of the g-index , 2006, Scientometrics.

[41]  Mike Thelwall,et al.  Determinants of research citation impact in nanoscience and nanotechnology , 2013, J. Assoc. Inf. Sci. Technol..

[42]  Jiawei Han,et al.  Citation Prediction in Heterogeneous Bibliographic Networks , 2012, SDM.

[43]  Lutz Bornmann,et al.  What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018 , 2019, Scientometrics.

[44]  F. Jalali,et al.  Artificial neural network back-calculation of flexible pavements with sensitivity analysis using Garson’s and connection weights algorithms , 2020, Innovative Infrastructure Solutions.

[45]  L. Freeman Centrality in social networks conceptual clarification , 1978 .

[46]  Nabil Amara,et al.  What can university administrators do to increase the publication and citation scores of their faculty members? , 2015, Scientometrics.

[47]  Abien Fred Agarap Deep Learning using Rectified Linear Units (ReLU) , 2018, ArXiv.

[48]  Cassidy R. Sugimoto,et al.  Bibliometrics: Global gender disparities in science , 2013, Nature.

[49]  Jian Wang,et al.  Citation time window choice for research impact evaluation , 2013, Scientometrics.

[50]  Joseph A. C. Delaney Sensitivity analysis , 2018, The African Continental Free Trade Area: Economic and Distributional Effects.

[51]  Daniel Svozil,et al.  Introduction to multi-layer feed-forward neural networks , 1997 .

[52]  Benjamin F. Jones,et al.  Atypical Combinations and Scientific Impact , 2013, Science.

[53]  Iman Tahamtan,et al.  Factors affecting number of citations: a comprehensive review of the literature , 2016, Scientometrics.

[54]  Andrea Giovanni Nuzzolese,et al.  Do altmetrics work for assessing research quality? , 2018, Scientometrics.

[55]  Vincent Larivière,et al.  The weakening relationship between the impact factor and papers' citations in the digital age , 2012, J. Assoc. Inf. Sci. Technol..

[56]  Chaomei Chen,et al.  Predictive effects of structural variation on citation counts , 2012, J. Assoc. Inf. Sci. Technol..

[57]  Vetle I. Torvik,et al.  Deconstructing the collaborative impact: Article and author characteristics that influence citation count , 2013, ASIST.

[58]  John Scott Social Network Analysis , 1988 .

[59]  Mike Thelwall,et al.  Could scientists use Altmetric.com scores to predict longer term citation counts? , 2018, J. Informetrics.

[60]  Karim R. Lakhani,et al.  Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science , 2016, Manag. Sci..

[61]  Vincent Larivière,et al.  The impact factor's Matthew Effect: A natural experiment in bibliometrics , 2009, J. Assoc. Inf. Sci. Technol..

[62]  Cai Yi Sensitivity Analysis , 2001 .

[63]  Mike Thelwall,et al.  Are Mendeley reader counts useful impact indicators in all fields? , 2017, Scientometrics.

[64]  Domingo Docampo,et al.  Measuring the academic reputation through citation networks via PageRank , 2018, J. Informetrics.

[65]  Mark B. Houston,et al.  The Impact of Article Method Type and Subject Area on Article Citations and Reference Diversity in JM, JMR, and JCR , 2001 .

[66]  Mike Thelwall,et al.  Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows , 2014, J. Assoc. Inf. Sci. Technol..

[67]  Rodrigo Costas,et al.  Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective , 2014, J. Assoc. Inf. Sci. Technol..

[68]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[69]  Ying Ding,et al.  Applying centrality measures to impact analysis: A coauthorship network analysis , 2009 .

[70]  M. Fenner What Can Article-Level Metrics Do for You? , 2013, PLoS biology.

[71]  Stephanie Valdivia,et al.  Determinants of the Index of Prices and Quotations on the Mexican Stock Exchange: Sensitivity Analysis Based on Artificial Neural Networks , 2016 .

[72]  L. Bornmann,et al.  Altmetrics and societal impact measurements: Match or mismatch? A literature review , 2020, El Profesional de la Información.

[73]  Fuyuki Yoshikane,et al.  Factors affecting citation rates of research articles , 2015, J. Assoc. Inf. Sci. Technol..

[74]  Zhenyu Wang,et al.  Which can better predict the future success of articles? Bibliometric indices or alternative metrics , 2019, Scientometrics.

[75]  Jingjing Zhang,et al.  The impact of a paper’s new combinations and new components on its citation , 2019, Scientometrics.

[76]  Lucila Ohno-Machado,et al.  Logistic regression and artificial neural network classification models: a methodology review , 2002, J. Biomed. Informatics.

[77]  Michael Schreiber,et al.  Restricting the h-index to a publication and citation time window: A case study of a timed Hirsch index , 2014, J. Informetrics.

[78]  Mike Thelwall,et al.  Which factors help authors produce the highest impact research? Collaboration, journal and document properties , 2013, J. Informetrics.

[79]  Christopher M. Snyder,et al.  Identifying the Effect of Open Access on Citations Using a Panel of Science Journals , 2013 .

[80]  Jenny Wooldridge,et al.  Altmetric scores: An early indicator of research impact , 2018, J. Assoc. Inf. Sci. Technol..