Performance-related differences of bibliometric statistical properties of research groups: Cumulative advantages and hierarchically layered networks

In this article we distinguish between top-performance and lower-performance groups in the analysis of statistical properties of bibliometric characteristics of two large sets of research groups. We find intriguing differences between top-performance and lower-performance groups, and between the two sets of research groups. These latter differences may indicate the influence of research management strategies. We report the following two main observations: First, lower-performance groups have a larger size-dependent cumulative advantage for receiving citations than top-performance groups. Second, regardless of performance, larger groups have fewer not-cited publications. Particularly for the lower-performance groups, the fraction of not-cited publications decreases considerably with size. We introduce a simple model in which processes at the microlevel lead to the observed phenomena at the macrolevel. Next, we fit our findings into the novel concept of hierarchically layered networks. In this concept, which provides the “infrastructure” for the model, a network of research groups constitutes a layer of one hierarchical step higher than the basic network of publications connected by citations. The cumulative size advantage of citations received by a group resembles preferential attachment in the basic network in which highly connected nodes (publications) increase their connectivity faster than less connected nodes. But in our study it is size that causes an advantage. In general, the larger a group (node in the research group network), the more incoming links this group acquires in a nonlinear, cumulative way. Nevertheless, top-performance groups are about an order of magnitude more efficient in creating linkages (i.e., receiving citations) than lower-performance groups. This implies that together with the size-dependent mechanism, preferential attachment, a quite common characteristic of complex networks, also works. Finally, in the framework of this study on performance-related differences of bibliometric properties of research groups, we also find that top-performance groups are, on average, more successful in the entire range of journal impact. © 2006 Wiley Periodicals, Inc.

[1]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[2]  M. Newman Clustering and preferential attachment in growing networks. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  A. Raan Measuring Science: Capita Selecta of Current Main Issues , 2004 .

[4]  J. S. Katz,et al.  Scale-independent indicators and research evaluation , 2000 .

[5]  Michel Zitt,et al.  Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalisation , 2005, Scientometrics.

[6]  J. S. Katz,et al.  The self-similar science system , 1999 .

[7]  A. Raan Statistical properties of bibliometric indicators: Research group indicator distributions and correlations , 2006 .

[8]  Anthony F. J. van Raan,et al.  Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods , 2005, Scientometrics.

[9]  Henk F. Moed,et al.  In basic science the percentage of “authoritative” references decreases as bibliographies become shorter , 2004, Scientometrics.

[10]  Anthony F. J. van Raan,et al.  Reference-based publication networks with episodic memories , 2003, Scientometrics.

[11]  Michel Zitt,et al.  Facing Diversity of Science: A Challenge for Bibliometric Indicators , 2005 .

[12]  R. Merton The Matthew Effect in Science , 1968, Science.

[13]  S. Redner How popular is your paper? An empirical study of the citation distribution , 1998, cond-mat/9804163.

[14]  A. F. J. Van Raan,et al.  Influence of interdisciplinarity on peer-review and bibliometric evaluations in physics research , 2001 .

[15]  J. S. Katz,et al.  Scale-Independent Bibliometric Indicators , 2005 .

[16]  S. N. Dorogovtsev,et al.  Evolution of networks , 2001, cond-mat/0106144.

[17]  Henk F. Moed,et al.  Improving the accuracy of Institute for Scientific Information's journal impact factors , 1995 .

[18]  Z. Neda,et al.  Measuring preferential attachment in evolving networks , 2001, cond-mat/0104131.

[19]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[20]  Albert-László Barabási,et al.  Hierarchical organization in complex networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[21]  Per Ottar Seglen,et al.  Causal relationship between article citedness and journal impact , 1994 .

[22]  R. Merton The Matthew Effect in Science, II: Cumulative Advantage and the Symbolism of Intellectual Property , 1988, Isis.

[23]  Per Ottar Seglen,et al.  The skewness of science , 1992 .

[24]  Anthony F. J. van Raan,et al.  Measurement of Central Aspects of Scientific Research: Performance, Interdisciplinarity, Structure , 2005 .

[25]  T. V. Leeuwen,et al.  Impact factors can mislead , 1996, Nature.

[26]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[27]  Ed J. Rinia,et al.  COMPARATIVE ANALYSIS OF A SET OF BIBLIOMETRIC INDICATORS AND CENTRAL PEER REVIEW CRITERIA. EVALUATION OF CONDENSED MATTER PHYSICS IN THE NETHERLANDS , 1998 .

[28]  Anthony F. J. van Raan,et al.  Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises , 1996, Scientometrics.

[29]  A. Vázquez Statistics of citation networks , 2001, cond-mat/0105031.

[30]  P. Lawrence The politics of publication , 2003, Nature.

[31]  Stefano Mossa,et al.  Truncation of power law behavior in "scale-free" network models due to information filtering. , 2002, Physical review letters.

[32]  A. Barabasi,et al.  Virtual Round Table on ten leading questions for network research , 2004 .

[33]  Michel Zitt,et al.  Correcting glasses help fair comparisons in international science landscape: Country indicators as a function of ISI database delineation , 2003, Scientometrics.

[34]  V. Eguíluz,et al.  Highly clustered scale-free networks. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[35]  A. C. Chiang Fundamental methods of mathematical economics , 1974 .

[36]  M. Newman Properties of highly clustered networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.