Effect of Peer Review on Citations in the Open Access Environment

Introduction Editorial peer review is widely used to select submissions to journals for publication and is presumed to improve their usefulness (Jefferson, Alderson, Wager & Davidoff, 2002). Publishing a peer reviewed article in a prestigious journal remains the highest validation for a work of scholarship. Peer review has served scholars well for centuries. The concept of reporting and validating research findings began in 1665, with the foundation of the Philosophical Transactions of the Royal Society of London. Peer review has enhanced the rigour and relevance of many scientific breakthroughs (Banks, 2006). Practically no historical accounts of the evolution of peer review exist. Contrary to common assumption, editorial peer review did not grow out of or interact with grant peer review. Editorial peer review procedures did not spread in an orderly way; they were not developed from editorial boards and passed on from journal to journal. Instead, casual referring out of articles on an individual basis may have occurred at any time, beginning in the early to mid-19th century (Burnham, 1990). Peer review is not perfect, and when it is done sloppily, journals publish research that is flawed. Even when peer review is rigorous, flawed research sometimes gets into the literature. Journals have long relied on peer review, yet concerns about its limitations have often been expressed. Critics point out that some reviewers are unqualified and others, because of personal or professional rivalry, are biased. Editors may even select reviewers on the basis of the reviewers' biases. Furthermore, two or more reviewers may have widely discrepant opinions about a study. Critics also make the point that peer review not only fails to prevent the publication of flawed research but also permits the publication of research that is fraudulent. Some have described peer review as arbitrary, subjective, and secretive. In addition, many critics (including some of the popular press) maintain that it is simply unnecessary and slows the communication of information to the public (Kassirer & Campion 1994). The present study makes an attempt to measure the effect of peer review process on research impact of publications in comparison to those which have not gone through the process via citation analysis. Objectives The following objectives are laid down for the study: * To assess the impact of peer review on citations * To compare the research impact of refereed and working papers Scope The scope of the present study is limited to research articles published in Information Research: An International Electronic Journal from 1998-2002. Hypothesis In view of the criticism of peer review process from various quarters, let us assume that it does not improve the quality of research output and consequently does not affect the research impact of publications and formulate the hypothesis "The research impact of refereed and working papers does not differ significantly" for the purpose of testing. Methodology Information Research: An International Electronic Journal is a high impact factor open access journal in the field of information science publishing working papers side by side with refereed articles. The publication output of five years (1998-2002) was selected for determining their research impact through citation analysis. The details of refereed and working papers were recorded separately across all the issues covering a time period of 1998-2002. All the 101 articles (74 refereed and 26 working papers) were searched in Scopus database for citations. Two articles (one refereed and one working paper) are not indexed by Scopus are thus not included in the study. The number of citations, self citations and other details were recorded for all the 99 articles. Standard statistical techniques were used to estimate various statistical tests. …