Editorial Judgments

Based on participant observation of editors’ decisions for a sociology journal, the paper investigates the peer review process. It shows a hidden interactivity in peer review, which is overlooked both by authors who impute social causes to unwelcome decisions, and by the preoccupation with ‘reliability’ prevalent in peer review research. This study shows that editorial judgments are: (1) attitudes taken by editorial readers toward various kinds of text, as a result of their membership in an intellectual milieu; (2) impressions gained through the reading process (through a ‘virtual interaction’ with the author); and (3) rationalizing statements about manuscripts made by editors and addressed to their peers on a committee. Since all these judgments are themselves subjected to judgments about their quality, the ‘review’ of peer review does not consist in an asymmetric examination of a text, but in the mutual monitoring of expert judgments, complementing and controlling, supervising and competing with each other. What has become known as scientific ‘criticism’ is an ongoing panoptic organization of communication: in peer review, judgments themselves are judged and made public.

[1]  Clark McPhail,et al.  Who Complains to Journal Editors and What Happens , 1986 .

[2]  Juan Miguel Campanario,et al.  Peer Review for Journals as it Stands Today—Part 2 , 1998 .

[3]  Stevan Harnad,et al.  Peer Commentary on Peer Review: A Case Study in Scientific Quality Control , 1983 .

[4]  Stuart A. Kirk,et al.  Recognition of Authors in Blind Review of Manuscripts , 1981 .

[5]  Harold Maurice Collins,et al.  New Light on Old Boys: Cognitive and Institutional Particularism in the Peer Review System , 1991 .

[6]  B. Latour La clef de Berlin : et autres leçons d'un amateur de sciences , 1993 .

[7]  M. Lynch The Discursive Production of Uncertainty , 1998 .

[8]  David L. Hull,et al.  Science as a Process: An Evolutionary Account of the Social and Conceptual Development of Science, David L. Hull. 1988. The University of Chicago Press, Chicago, IL. 608 pages. ISBN: 0-226-35060-4. $39.95 , 1989 .

[9]  Michael Lynch,et al.  Art and artifact in laboratory science , 1985 .

[10]  Clark McPhail,et al.  The Manuscript Review and Decision-Making Process , 1987 .

[11]  Juan Miguel Campanario,et al.  Peer Review for Journals as it Stands Today—Part 1 , 1998 .

[12]  Michael Lynch,et al.  Art and Artifact in Laboratory Science: A Study of Shop Work and Shop Talk in a Research Laboratory , 1985 .

[13]  A. Yankauer,et al.  How blind is blind review? , 1991, American journal of public health.

[14]  J. Armstrong,et al.  Peer review for journals: Evidence on quality control, fairness, and innovation , 1997 .

[15]  R. Fletcher,et al.  The effects of blinding on the quality of peer review. A randomized trial. , 1990, JAMA.

[16]  H. Sahner,et al.  Zur Lage der Soziologie. Zur Selektivität von Herausgebern: Eine Input-output-Analyse der „Zeitschrift für Soziologie“ , 1982 .

[17]  L. Hargens,et al.  Are sociologists’ publications uncited? Citation rates of journal articles, chapters, and books , 1991 .

[18]  Ann E. Reisner,et al.  Gatekeeping in action : Editorial conferences and assessments of newsworthiness , 1998 .

[19]  R. Merton,et al.  Patterns of evaluation in science: Institutionalisation, structure and functions of the referee system , 1971 .

[20]  D. Cicchetti The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation , 1991, Behavioral and Brain Sciences.

[21]  T. Scheffer Asylgewährung: Eine ethnographische Verfahrensanalyse , 2001 .

[22]  Michèle Lamont,et al.  What is Originality in the Humanities and the Social Sciences? , 2004 .

[23]  D. W. Fiske,et al.  But the Reviewers Are Making Different Criticisms of My Paper! Diversity and Uniqueness in Reviewer Comments. , 1990 .

[24]  Lee Shiflett,et al.  A difficult balance: Editorial peer review in medicine , 1988, J. Am. Soc. Inf. Sci..

[25]  Erwin Parfy,et al.  Mahoney, Michael J. , 2005 .

[26]  Clark McPhail,et al.  MIXED MESSAGES: Referees' Comments on the Manuscripts They Review , 1989 .

[27]  Stevan Harnad Learned inquiry and the Net: the role of peer review, peer commentary and copyright , 1998, Learn. Publ..

[28]  J. M. Brittain Peer commentary on peer review: A case study in scientific quality control: S. Harnad (Editor). Cambridge University Press, 1982. 71 pp. ISBN 0 521 27306 4 , 1985 .

[29]  Stefan Hirschauer Peer Review Verfahren auf dem Prüfstand / Peer Review Research – Reviewed , 2004 .

[30]  Henry E. Kyburg,et al.  Science as Process. , 1993 .

[31]  I. Hartmann,et al.  Peer review at the Deutsche Forschungsgemeinschaft , 1990, Scientometrics.

[32]  Greg Myers,et al.  Texts as Knowledge Claims: The Social Construction of Two Biology Articles , 1985 .

[33]  S. Ceci,et al.  Peer-review practices of psychological journals: The fate of published articles, submitted again , 1982, Behavioral and Brain Sciences.

[34]  R. Blank The Effects of Double-Blind versus Single-Blind Reviewing: Experimental Evidence from The American Economic Review , 1991 .

[35]  S. Lilley Conspiracy, What Conspiracy?: Social Science, Funding and the Politics of Accusation , 2000 .

[36]  M. Mahoney Publication prejudices: An experimental study of confirmatory bias in the peer review system , 1977, Cognitive Therapy and Research.

[37]  Stevan Harnad,et al.  Rational Disagreement in Peer Review , 1985 .

[38]  M. Lamont,et al.  What is Originality in the Social Sciences and the Humanities , 2004 .

[39]  Michael J. Moravcsik,et al.  Patterns of evaluation in science , 1971 .