Exploiting Data Reliability and Fuzzy Clustering for Journal Ranking

Journal impact indicators are widely accepted as possible measurements of academic journal quality. However, much debate has recently surrounded their use, and alternative journal impact evaluation techniques are desirable. Aggregation of multiple indicators offers a promising method to produce a more robust ranking result, avoiding the possible bias caused by the use of a single impact indicator. In this paper, fuzzy aggregation and fuzzy clustering, especially the ordered weighted averaging (OWA) operators are exploited to aggregate the quality scores of academic journals that are obtained from different impact indicators. Also, a novel method for linguistic term-based fuzzy cluster grouping is proposed to rank academic journals. The paper allows for the construction of distinctive fuzzy clusters of academic journals on the basis of their performance with respect to different journal impact indicators, which may be subsequently combined via the use of the OWA operators. Journals are ranked in relation to their memberships in the resulting combined fuzzy clusters. In particular, the nearest-neighbor guided aggregation operators are adopted to characterize the reliability of the indicators, and the fuzzy clustering mechanism is utilized to enhance the interpretability of the underlying ranking procedure. The ranking results of academic journals from six subjects are systematically compared with the outlet ranking used by the Excellence in Research for Australia, demonstrating the significant potential of the proposed approach.

[1]  Gleb Beliakov,et al.  Aggregation Functions: A Guide for Practitioners , 2007, Studies in Fuzziness and Soft Computing.

[2]  Tossapon Boongoen,et al.  Fuzzy Orders-of-Magnitude-Based Link Analysis for Qualitative Alias Detection , 2012, IEEE Transactions on Knowledge and Data Engineering.

[3]  J. Malczewski,et al.  Ordered weighted averaging with fuzzy quantifiers: GIS-based multicriteria evaluation for land-use suitability analysis , 2006 .

[4]  Gleb Beliakov,et al.  Using Choquet integrals for kNN approximation and classification , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[5]  Pan Su,et al.  Link-based approach for bibliometric journal ranking , 2013, Soft Comput..

[6]  R. Mesiar,et al.  Aggregation operators: new trends and applications , 2002 .

[7]  Tossapon Boongoen,et al.  Clus-DOWA: A new dependent OWA operator , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[8]  James M. Keller,et al.  Comparing Fuzzy, Probabilistic, and Possibilistic Partitions , 2010, IEEE Transactions on Fuzzy Systems.

[9]  Johannes Stegmann,et al.  How to evaluate journal impact factors , 1997, Nature.

[10]  Tossapon Boongoen,et al.  Nearest-Neighbor Guided Evaluation of Data Reliability and Its Applications , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Gleb Beliakov,et al.  Citation-based journal ranks: The use of fuzzy measures , 2011, Fuzzy Sets Syst..

[12]  Ronald R. Yager,et al.  On ordered weighted averaging aggregation operators in multicriteria decisionmaking , 1988, IEEE Trans. Syst. Man Cybern..

[13]  R. Yager.  Families of OWA operators , 1993 .

[14]  Bertrand Meyer,et al.  ViewpointResearch evaluation for computer science , 2009, CACM.

[15]  Ian R. Dobson,et al.  Using Data and Experts to Make the Wrong Decision , 2014 .

[16]  Johannes Stegmann,et al.  Citation rates, knowledge export and international visibility of dermatology journals listed and not listed in theJournal Citation Reports , 2001, Scientometrics.

[17]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[18]  E. Garfield.  The history and meaning of the journal impact factor. , 2006, JAMA.

[19]  Anna Poletti,et al.  The new ERA of journal ranking: The consequences of Australia's fraught encounter with 'quality' , 2011 .

[20]  Pan Su,et al.  Feature Selection Ensemble , 2012, Turing-100.

[21]  Gregory W. Corder,et al.  Nonparametric Statistics : A Step-by-Step Approach , 2014 .

[22]  Joydeep Ghosh,et al.  CONSENSUS-BASED ENSEMBLES OF SOFT CLUSTERINGS , 2008, MLMTA.

[23]  Claus Rinner,et al.  Web-enabled spatial decision analysis using Ordered Weighted Averaging (OWA) , 2002, J. Geogr. Syst..

[24]  Carl T. Bergstrom,et al.  The Eigenfactor™ Metrics , 2008, The Journal of Neuroscience.

[25]  Joydeep Ghosh,et al.  Data Clustering Algorithms And Applications , 2013 .

[26]  John F. Kolen,et al.  Reducing the time complexity of the fuzzy c-means algorithm , 2002, IEEE Trans. Fuzzy Syst..

[27]  F. Bartolucci,et al.  Ranking scientific journals via latent class models for polytomous item response data , 2015 .

[28]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[29]  Pan Su,et al.  Link-based pairwise similarity matrix approach for fuzzy c-means clustering ensemble , 2014, 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[30]  Tossapon Boongoen,et al.  Comparative study of matrix refinement approaches for ensemble clustering , 2013, Machine Learning.

[31]  Ronald R. Yager,et al.  Generalized OWA Aggregation Operators , 2004, Fuzzy Optim. Decis. Mak..

[32]  Johan Bollen,et al.  Toward alternative metrics of journal impact: A comparison of download and citation data , 2005, Inf. Process. Manag..

[33]  S. Ayatollahi,et al.  Factors Affecting Journal Quality Indicator in Scopus (SCImago Journal Rank) in Obstetrics and Gynecology Journals: a Longitudinal Study (1999-2013) , 2014, Acta informatica medica : AIM : journal of the Society for Medical Informatics of Bosnia & Herzegovina : casopis Drustva za medicinsku informatiku BiH.

[34]  Qiang Shen,et al.  Feature Selection With Harmony Search , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[35]  Ronald Rousseau,et al.  Journal Evaluation: Technical and Practical Issues, , 2002, Libr. Trends.

[36]  Clyde W. Holsapple,et al.  A new map for knowledge dissemination channels , 2009, CACM.

[37]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[38]  Jozo J. Dujmovic,et al.  Properties of Local Andness/Orness , 2007, IFSA.

[39]  Loet Leydesdorff,et al.  Can scientific journals be classified in terms of aggregated journal-journal citation relations using the Journal Citation Reports? , 2009, J. Assoc. Inf. Sci. Technol..

[40]  Zeshui Xu,et al.  Dependent OWA Operators , 2006, MDAI.

[41]  Ronald R. Yager,et al.  Using Stress Functions to Obtain OWA Operators , 2007, IEEE Transactions on Fuzzy Systems.