Estimation of 'face validity' of psychological research papers based on in-text citations

The possibility of evaluation of the quality of psychological papers on the basis of in-text citation was examined. In addition to traditional indices of this kind the measures of degree of overlapping, based on co-citations in various parts of the articles, were also used. The rationale for their use was found in author's obligation to cite the same sources in different parts of the article, so as to demonstrate consistency of the study. By using the series of one-way ANOVA-s and the discriminant analysis it was tested to what degree citation measures differentiate among three groups of articles of various quality. One of the groups involved papers published in distinguished international journals, while the other two articles from Yugoslav journals, classified according to the academic status of heir authors. The results showed that citation-based measures successfully differentiate among three groups, especially the group of papers of international quality. Accordingly, the results open the possibility of using these indicators as measures of face validity in preliminary automatic positive selection of papers submitted to publication.

[1]  D. Edge.  Quantitative Measures of Communication in Science: A Critical Review , 1979, History of science; an annual review of literature, research and teaching.

[2]  Terrence A. Brooks,et al.  Private acts and public objects: An investigation of citer motivations , 1985, J. Am. Soc. Inf. Sci..

[3]  S. Cozzens.  Comparing the Sciences: Citation Context Analysis of Papers from Neuropharmacology and the Sociology of Science , 1985 .

[4]  J. King.  A review of bibliometric and other science indicators and their role in research evaluation , 1987, J. Inf. Sci..

[5]  W. Shadish,et al.  Author Judgements about Works They Cite: Three Studies from Psychology Journals , 1995 .

[6]  Susan E. Cozzens,et al.  What do citations count? the rhetoric-first model , 1989, Scientometrics.

[7]  Alan L. Porter,et al.  Citations and scientific progress: Comparing bibliometric measures with scientist judgments , 1988, Scientometrics.

[8]  J.-L. Sikorav,et al.  The utility of scientific papers , 1991, Scientometrics.

[9]  S. D. Haitun,et al.  Problems of quantitative analysis of scientific activities: The non-additivity of data. Part I , 1986, Scientometrics.

[10]  Sven Hemlin,et al.  Scientific quality in the eyes of the scientist. A questionnaire study , 1993, Scientometrics.

[11]  Anthony F. J. van Raan,et al.  A validation study of bibliometric indicators: The comparative performance of cum laude doctorates in chemistry , 1989, Scientometrics.

[12]  Michael J. Moravcsik,et al.  Applied scientometrics: An assessment methodology for developing countries , 1985, Scientometrics.