The reliability of network density and composition measures

Abstract Many measures based on egocentric network data, such as age composition or (local) network density, can be viewed as ‘aggregate’ measures: they are mean values of the alter attributes or the dyadic attributes that fall within a given respondent's egocentric network. Internal consistency methods of classical test theory are not suitable for assessing the reliability of such measures: such methods presume a ‘crossed’ design for data collection in which each respondent is scored on the same set of indicators. In designs for gathering egocentric network data, alters are instead ‘nested’ within respondents; moreover the number of alters may differ across respondents. This paper evaluates the reliability of composition and density measures via analysis-of-variance approaches to reliability known as generalizability theory. Reliability estimates are presented for egocentric measures based on the 1985, 1987, and 1988 General Social Surveys and for the 1977–1978 Northern California Community Study. Ethnoreligious composition, political composition, density, and composition of a network by ‘friends’ or co-members of organizations are measured with relatively high reliability, even for a relatively small number of alters. Other measures require more alters to attain adequate reliability, and some, such as sex composition, remain problematic even when the number of alters grows quite large. The sensitivity of reliability estimates to differences in instrument design is examined using design variations in the surveys studied.

[1]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[2]  C. Fischer.  What Do We Mean by 'Friend'? An Inductive Study* , 1982 .

[3]  Richard J. Shavelson,et al.  Generalizability Theory: A Primer , 1991 .

[4]  S. K. Mitchell.  Interobserver agreement, reliability, and generalizability of data collected in observational studies. , 1979 .

[5]  Walter Bien,et al.  Using egocentered networks in survey research. A methodological preview on an application of social network analysis in the area of family research , 1991 .

[6]  R. Alba,et al.  Bonds of Pluralism: The Form and Substance of Urban Social Networks. , 1974 .

[7]  P. V. Marsden,et al.  Homogeneity in confiding relations , 1988 .

[8]  Seymour Sudman,et al.  Experiments in the measurement of the size of social networks , 1985 .

[9]  Tom W. Smith.  Classifying Protestant Denominations , 1990 .

[10]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[11]  Ronald S. Burt,et al.  Network items and the general social survey , 1984 .

[12]  P. V. Marsden,et al.  Core Discussion Networks of Americans , 1987 .

[13]  Robert M. O'Brien,et al.  Estimating the Reliability of Aggregate-Level Variables Based on Individual-Level Characteristics , 1990 .

[14]  C. Fischer,et al.  To Dwell among Friends: Personal Networks in Town and City. , 1984 .

[15]  D. Knoke.  Networks of Political Action: Toward Theory Construction , 1990 .

[16]  J. Mitchell,et al.  The Concept and Use of Social Networks , 1969 .

[17]  Ronald S. Burt,et al.  A note on sociometric order in the general social survey network data , 1986 .

[18]  P. V. Marsden,et al.  Measuring Tie Strength , 1984 .

[19]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[20]  Noah E. Friedkin.  A Guttman scale for the strength of an interpersonal tie , 1990 .