Quality Assessment : An Integrated Subjective and Objective Approach

Many universities, research institutions, and government funding agencies are continuously attempting to grade or rank journals for their academic value. Such grading is needed both for funding and resource allocation purposes and for personnel decisions. In grading journals, objective information, such as the impact factors (the most commonly used journal citation information provided by Journal Citation Reports) and/or subjective information, such as experts' judgments about the journals, are used. Grading of journals is typically done by a committee. Most of the existing approaches only consider either the subjective or the objective information. While such approaches work in many cases, they may not work in many others. Complaints about politics, inconsistencies and unfairness are common in many situations. This paper presents a methodology that integrates both types of information, providing a comprehensive method for assessing the quality of journals. The method employs a fuzzy set approach that also deals with the imprecise and missing information frequently inherited in the evaluation process. The proposed evaluation model is implemented in a Web-based system. A pilot evaluation study indicated that participants clearly preferred the system and its major features.