Identifying Anomalous Citations for Objective Evaluation of Scholarly Article Impact

Evaluating the impact of a scholarly article is of great significance and has attracted great attentions. Although citation-based evaluation approaches have been widely used, these approaches face limitations e.g. in identifying anomalous citations patterns. This negligence would inevitably cause unfairness and inaccuracy to the article impact evaluation. In this study, in order to discover the anomalous citations and ensure the fairness and accuracy of research outcome evaluation, we investigate the citation relationships between articles using the following factors: collaboration times, the time span of collaboration, citing times and the time span of citing to weaken the relationship of Conflict of Interest (COI) in the citation network. Meanwhile, we study a special kind of COI, namely suspected COI relationship. Based on the COI relationship, we further bring forward the COIRank algorithm, an innovative scheme for accurately assessing the impact of an article. Our method distinguishes the citation strength, and utilizes PageRank and HITS algorithms to rank scholarly articles comprehensively. The experiments are conducted on the American Physical Society (APS) dataset. We find that about 80.88% articles contain contributed citations by co-authors in 26,366 articles and 75.55% articles among these articles are cited by the authors belonging to the same affiliation, indicating COI and suspected COI should not be ignored for evaluating impact of scientific papers objectively. Moreover, our experimental results demonstrate COIRank algorithm significantly outperforms the state-of-art solutions. The validity of our approach is verified by using the probability of Recommendation Intensity.

[1]  E. Garfield.  The history and meaning of the journal impact factor. , 2006, JAMA.

[2]  Lise Getoor,et al.  FutureRank: Ranking Scientific Articles by Predicting their Future PageRank , 2009, SDM.

[3]  Shivendra Singh,et al.  Impact of Self-Citations on Impact Factor: A Study Across Disciplines, Countries and Continents , 2015 .

[4]  Xiaojun Wan,et al.  Are all literature citations equally important? Automatic citation strength estimation and its applications , 2014, J. Assoc. Inf. Sci. Technol..

[5]  Heyan Huang,et al.  Tri-Rank: An Authority Ranking Framework in Heterogeneous Academic Networks by Mutual Reinforce , 2014, 2014 IEEE 26th International Conference on Tools with Artificial Intelligence.

[6]  James A. Evans,et al.  Open Access and Global Participation in Science , 2009, Science.

[7]  Sergei Maslov,et al.  Ranking scientific publications using a model of network traffic , 2006, ArXiv.

[8]  Feng Xia,et al.  MVCWalker: Random Walk-Based Most Valuable Collaborators Recommendation Exploiting Academic Factors , 2014, IEEE Transactions on Emerging Topics in Computing.

[9]  L. Egghe,et al.  Theory and practise of the g-index , 2006, Scientometrics.

[10]  Hongyuan Zha,et al.  Co-ranking Authors and Documents in a Heterogeneous Network , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[11]  Menghui Li,et al.  Quantifying the influence of scientists and their publications: distinguishing between prestige and popularity , 2011, ArXiv.

[12]  Xiaoming Zhang,et al.  Future Influence Ranking of Scientific Literature , 2014, SDM.

[13]  Linyuan Lu,et al.  Quantifying the influence of scientists and their publications: Distinguish prestige from popularity , 2011, ArXiv.

[14]  Ming Zeng,et al.  Ranking Scientific Articles by Exploiting Citations, Authors, Journals, and Time Information , 2013, AAAI.

[15]  Chun-Ting Zhang,et al.  A novel triangle mapping technique to study the h-index based citation distribution , 2013, Scientific Reports.

[16]  Feng Xia,et al.  PNCOIRank: Evaluating the Impact of Scholarly Articles with Positive and Negative Citations , 2016, WWW.

[17]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[18]  An Zeng,et al.  Ranking scientific publications: the effect of nonlinearity , 2014, Scientific Reports.

[19]  Hai Zhuge,et al.  Towards an effective and unbiased ranking of scientific literature through mutual reinforcement , 2012, CIKM.

[20]  Giulio Cimini,et al.  Removing spurious interactions in complex networks , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[21]  A. D. Jackson,et al.  Measures for measures , 2006, Nature.

[22]  John P A Ioannidis,et al.  A generalized view of self-citation: direct, co-author, collaborative, and coercive induced self-citation. , 2015, Journal of psychosomatic research.

[23]  D. Kallmes,et al.  Impact of self-citation on the H index in the field of academic radiology. , 2012, Academic radiology.

[24]  Frank-Thorsten Krell,et al.  Losing the numbers game: abundant self-citations put journals at risk for a life without an impact factor , 2014 .

[25]  Oren Etzioni,et al.  Identifying Meaningful Citations , 2015, AAAI Workshop: Scholarly Big Data.

[26]  Alan Fersht,et al.  The most influential journals: Impact Factor and Eigenfactor , 2009, Proceedings of the National Academy of Sciences.