Selecting manuscripts for a high-impact journal through peer review: A citation analysis of communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhere

All journals that use peer review have to deal with the following question: Does the peer review system fulfill its declared objective to select the “best” scientific work? We investigated the journal peer-review process at Angewandte Chemie International Edition (AC-IE), one of the prime chemistry journals worldwide, and conducted a citation analysis for Communications that were accepted by the journal (n = 878) or rejected but published elsewhere (n = 959). The results of negative binomial-regression models show that holding all other model variables constant, being accepted by AC-IE increases the expected number of citations by up to 50p. A comparison of average citation counts (with 95p confidence intervals) of accepted and rejected (but published elsewhere) Communications with international scientific reference standards was undertaken. As reference standards, (a) mean citation counts for the journal set provided by Thomson Reuters corresponding to the field “chemistry” and (b) specific reference standards that refer to the subject areas of Chemical Abstracts were used. When compared to reference standards, the mean impact on chemical research is for the most part far above average not only for accepted Communications but also for rejected (but published elsewhere) Communications. However, average and below-average scientific impact is to be expected significantly less frequently for accepted Communications than for rejected Communications. All in all, the results of this study confirm that peer review at AC-IE is able to select the “best” scientific work with the highest impact on chemical research. © 2008 Wiley Periodicals, Inc.

[1]  R. Merton.  The Matthew Effect in Science , 1968, Science.

[2]  J. Wilson.  Peer review and publication. Presidential address before the 70th annual meeting of the American Society for Clinical Investigation, San Francisco, California, 30 April 1978. , 1978, The Journal of clinical investigation.

[3]  B. Martin,et al.  Some partial indicators of scientific progress in radio astronomy , 1983 .

[4]  S. Lock,et al.  A difficult balance: editorial peer review in medicine continued. , 1985 .

[5]  S. Cozzens.  Comparing the Sciences: Citation Context Analysis of Papers from Neuropharmacology and the Sociology of Science , 1985 .

[6]  Robert A. Stine,et al.  An Introduction to Bootstrap Methods , 1989 .

[7]  D. Hosmer,et al.  Applied Logistic Regression , 1991 .

[8]  Blaise Cronin,et al.  The trajectory of rejection , 1992, J. Documentation.

[9]  Hans-Dieter Daniel,et al.  Guardians of Science: Fairness and Reliability of Peer Review , 1994 .

[10]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[11]  Ronald N. Kostoff,et al.  The principles and practices of peer review , 1997 .

[12]  P. Kline.  The New Psychometrics: Science, Psychology and Measurement , 1998 .

[13]  T. Opthof,et al.  Regrets or no regrets? No regrets! The fate of rejected manuscripts. , 2000, Cardiovascular research.

[14]  A. Weller.  Editorial peer review : its strengths and weaknesses , 2001 .

[15]  Tx Station.  Stata Statistical Software: Release 7. , 2001 .

[16]  H. P. Dalen,et al.  Attention and the art of scientific publishing , 2001 .

[17]  J. S. Long,et al.  Regression models for categorical dependent variables using Stata, 2nd Edition , 2005 .

[18]  Katherine M. Whitley,et al.  Analysis of Scifinder Scholar and Web of Science citation searches , 2002, J. Assoc. Inf. Sci. Technol..

[19]  R. Wears,et al.  Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. , 2002, JAMA.

[20]  Brian Everitt,et al.  A Handbook of Statistical Analyses Using SPSS , 2003 .

[21]  T. Braun.  Keeping the Gates of Science Journals , 2004 .

[22]  A. Raan.  Measuring Science: Capita Selecta of Current Main Issues , 2004 .

[23]  L. Philip Schumm.  Review of Data Analysis Using Stata by Kohler and Kreuter , 2005 .

[24]  Frauke Kreuter,et al.  Data Analysis Using Stata , 2005 .

[25]  T. W. Dewitt,et al.  Science citation index and chemistry , 1980, Scientometrics.

[26]  Peter Vinkler,et al.  Evaluation of some methods for the relative assessment of scientific publications , 1986, Scientometrics.

[27]  Wolfgang Glänzel,et al.  A characterization of scientometric distributions based on harmonic means , 2005, Scientometrics.

[28]  P. Gölitz.  Who is going to read all this? , 2005, Angewandte Chemie.

[29]  Hans-Dieter Daniel,et al.  Publications as a measure of scientific advancement and of scientists' productivity , 2005, Learn. Publ..

[30]  J. Koricheva,et al.  What determines the citation frequency of ecological papers? , 2005, Trends in ecology & evolution.

[31]  H. Herbertz,et al.  Does it pay to cooperate? A bibliometric case study in molecular biology , 1995, Scientometrics.

[32]  Lutz Bornmann,et al.  Selecting scientific excellence through committee peer review - A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicants , 2006, Scientometrics.

[33]  R. Rousseau.  Temporal differences in self-citation rates of scientific journals , 2006, Scientometrics.

[34]  C. Jennings.  Quality and value: The true purpose of peer review , 2006 .

[35]  Grant Lewison,et al.  The effect of funding on the outputs of biomedical research , 2006, Scientometrics.

[36]  Anthony F. J. van Raan,et al.  Advanced bibliometric methods for the evaluation of universities , 1999, Scientometrics.

[37]  Koenraad Debackere,et al.  A concise review on the role of author self-citations in information science, bibliometrics and science policy , 2006, Scientometrics.

[38]  Peter Vinkler,et al.  Relations of relative scientometric impact indicators. The relative publication strategy index , 1997, Scientometrics.

[39]  Brian Everitt,et al.  A Handbook of Statistical Analyses Using Stata , 2006 .

[40]  Thed N. van Leeuwen,et al.  Modelling of bibliometric approaches and importance of output verification in research performance assessment , 2007 .

[41]  Benjamin F. Jones,et al.  Supporting Online Material Materials and Methods Figs. S1 to S3 References the Increasing Dominance of Teams in Production of Knowledge , 2022 .

[42]  Iain D. Craig,et al.  Do open access articles have greater citation impact?: A critical review of the literature , 2007, J. Informetrics.

[43]  D. Greenwood,et al.  Selective attrition and bias in a longitudinal health survey among survivors of a disaster , 2007, BMC medical research methodology.

[44]  L. Bornmann,et al.  Multiple publication on a single research study: Does it pay? The influence of number of research articles on total citation counts in biomedicine , 2007 .

[45]  L. Bornmann,et al.  Functional use of frequently and infrequently cited articles in citing publications: A content analysis of citations to articles with low and high citation counts , 2007 .

[46]  Lutz Bornmann,et al.  Convergent validation of peer review decisions using the h index: Extent of and reasons for type I and type II errors , 2007, J. Informetrics.

[47]  K. A. McKibbon,et al.  Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study , 2008, BMJ : British Medical Journal.

[48]  L. Bornmann,et al.  The effectiveness of the peer review process: inter-referee agreement and predictive validity of manuscript refereeing at Angewandte Chemie. , 2008, Angewandte Chemie.

[49]  Lutz Bornmann,et al.  OPEN PEN ACCESS CCESS , 2008 .

[50]  H. Marsh,et al.  Improving the Peer-review Process for Grant Applications , 2022 .

[51]  Lutz Bornmann,et al.  What do citation counts measure? A review of studies on citing behavior , 2008, J. Documentation.

[52]  Stevan Harnad,et al.  Open access scientometrics and the UK Research Assessment Exercise , 2007, Scientometrics.

[53]  Loet Leydesdorff,et al.  Citation Environment of Angewandte Chemie , 2007, ArXiv.

[54]  Christoph Neuhaus,et al.  A new reference standard for citation analysis in chemistry and related fields based on the sections of Chemical Abstracts , 2009 .