Social impact assessment of scientist from mainstream news and weblogs

Research policy makers, funding agencies, universities and government organizations evaluate research output or impact based on the traditional citation count, peer review, h-index and journal impact factors. These impact measures also known as bibliometric indicators are limited to the academic community and cannot provide the broad perspective of research impact in public, government or business. The understanding that scholarly impact outside scientific and academic sphere has given rise to an area of scientometrics called alternative metrics or “altmetrics.” Moreover, researchers in this area incline to center around gauging scientific activity via social media, namely Twitter. However, these count-based measurements of impact are sensitive to gaming as they lack concrete references to the primary source. In this work, we expand a conventional citation graph to a heterogeneous graph of publications, scientists, venues, organizations based on more reliable social media sources such as mainstream news and weblogs. Our method is composed of two components: the first one is combining the bibliometric data with social media data like blogs and mainstream news. The second component investigates how standard graph-based metrics can be applied to a heterogeneous graph to predict the academic impact. Our result showed moderate correlations and positive associations between the computed graph-based metrics with academic impact and also reasonably predict the academic impact of researchers.

[1]  H. Staras,et al.  Spectrum conservation in the Land Mobile Radio Services , 1971, IEEE Spectrum.

[2]  Hongyuan Zha,et al.  Co-ranking Authors and Documents in a Heterogeneous Network , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[3]  Melissa Goertzen,et al.  Wired Academia: Why Social Science Scholars Are Using Social Media , 2013, 2013 46th Hawaii International Conference on System Sciences.

[4]  Kevin D. O'Brien Communicating orthodontic research via social media , 2016 .

[5]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[6]  Cassidy R. Sugimoto,et al.  Do Altmetrics Work? Twitter and Ten Other Social Web Services , 2013, PloS one.

[7]  Carl T. Bergstrom Eigenfactor Measuring the value and prestige of scholarly journals , 2007 .

[8]  Mike Thelwall,et al.  Bibliometrics to webometrics , 2008, J. Inf. Sci..

[9]  Leo Egghe,et al.  Dynamic h-index: The Hirsch index in function of time , 2007, J. Assoc. Inf. Sci. Technol..

[10]  Paul Groth,et al.  The Altmetrics Collection , 2012, PloS one.

[11]  Daniel Lemire,et al.  Measuring academic influence: Not all citations are equal , 2015, J. Assoc. Inf. Sci. Technol..

[12]  Ludo Waltman,et al.  F1000 recommendations as a new data source for research evaluation: A comparison with citations , 2013, ArXiv.

[13]  Victor M Montori,et al.  Use of Web 2.0 Social Media Platforms to Promote Community-Engaged Research Dialogs: A Preliminary Program Evaluation , 2016, JMIR research protocols.

[14]  E. Garfield Citation analysis as a tool in journal evaluation. , 1972, Science.

[15]  Isabell M. Welpe,et al.  I Like, I Cite? Do Facebook Likes Predict the Impact of Scientific Work? , 2015, PloS one.

[16]  Brian Davis,et al.  On Developing Extraction Rules for Mining Informal Scientific References from Altmetric Data Sources , 2015, NLDB.

[17]  Carl T. Bergstrom,et al.  The Eigenfactor™ Metrics , 2008, The Journal of Neuroscience.

[18]  Tim S. Evans,et al.  Ranking Journals Using Altmetrics , 2015, ISSI.

[19]  Ming Zeng,et al.  Ranking Scientific Articles by Exploiting Citations, Authors, Journals, and Time Information , 2013, AAAI.

[20]  Gunther Eysenbach,et al.  Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact , 2011, Journal of medical Internet research.

[21]  Alexander M. Petersen,et al.  Inequality and cumulative advantage in science careers: a case study of high-impact journals , 2014, EPJ Data Science.

[22]  Konrad Paul Kording,et al.  Future impact: Predicting scientific success , 2012, Nature.

[23]  Denis Gillet,et al.  Identifying influential scholars in academic social media platforms , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).

[24]  Bradley M. Hemminger,et al.  Altmetrics in the wild: Using social media to explore scholarly impact , 2012, ArXiv.

[25]  Sarah de Rijcke,et al.  Quantified academic selves: the gamification of research through social networking services , 2016, Inf. Res..

[26]  Johan Bollen,et al.  Co-authorship networks in the digital library research community , 2005, Inf. Process. Manag..

[27]  Vincent Larivière,et al.  Who reads research articles? An altmetrics analysis of Mendeley user categories , 2015, J. Assoc. Inf. Sci. Technol..

[28]  Paul Mcfedries Measuring the impact of altmetrics [Technically Speaking] , 2012 .

[29]  Taha Yasseri,et al.  The distorted mirror of Wikipedia: a quantitative analysis of Wikipedia coverage of academics , 2013, EPJ Data Science.

[30]  Diana Maynard,et al.  JAPE: a Java Annotation Patterns Engine , 2000 .

[31]  Stasa Milojevic,et al.  Accuracy of simple, initials-based methods for author name disambiguation , 2013, J. Informetrics.

[32]  Rodrigo Costas,et al.  How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications , 2014, Scientometrics.

[33]  J. H. Steiger Tests for comparing elements of a correlation matrix. , 1980 .

[34]  Roberta Kwok,et al.  Research impact: Altmetrics make their mark , 2013, Nature.

[35]  Charles Elkan,et al.  An Efficient Domain-Independent Algorithm for Detecting Approximately Duplicate Database Records , 1997, DMKD.

[36]  Ralph A. Szweda,et al.  Information processing management , 1972 .

[37]  D. Bonett,et al.  Sample size requirements for estimating pearson, kendall and spearman correlations , 2000 .

[38]  Lise Getoor,et al.  FutureRank: Ranking Scientific Articles by Predicting their Future PageRank , 2009, SDM.

[39]  Hamish Cunningham,et al.  GATE-a General Architecture for Text Engineering , 1996, COLING.

[40]  Amin Mazloumian,et al.  Predicting Scholars' Scientific Impact , 2012, PloS one.

[41]  Leo Katz,et al.  A new status index derived from sociometric analysis , 1953 .

[42]  James Caverlee,et al.  PageRank for ranking authors in co-citation networks , 2009, J. Assoc. Inf. Sci. Technol..

[43]  Brian Davis,et al.  Towards predicting academic impact from mainstream news and weblogs: A heterogeneous graph based approach , 2016, 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[44]  Dana Ron,et al.  Algorithmic Stability and Sanity-Check Bounds for Leave-One-Out Cross-Validation , 1997, Neural Computation.

[45]  Gustavo Lannelongue,et al.  Scholarly Impact Revisited , 2012 .

[46]  Christopher D. Manning,et al.  Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling , 2005, ACL.

[47]  Ingo Scholtes,et al.  Predicting scientific success based on coauthorship networks , 2014, EPJ Data Science.

[48]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[49]  Christian Pieter Hoffmann,et al.  Impact Factor 2.0: Applying Social Network Analysis to Scientific Impact Assessment , 2014, 2014 47th Hawaii International Conference on System Sciences.

[50]  C. Neylon,et al.  Article-Level Metrics and the Evolution of Scientific Impact , 2009, PLoS biology.

[51]  Henk F. Moed,et al.  Citation Analysis in Research Evaluation , 1899 .

[52]  Ludo Waltman,et al.  F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations , 2014, J. Assoc. Inf. Sci. Technol..

[53]  Sergey Brin,et al.  Reprint of: The anatomy of a large-scale hypertextual web search engine , 2012, Comput. Networks.