Question-order effects in social network name generators

Abstract Social network surveys are an important tool for empirical research in a variety of fields, including the study of social capital and the evaluation of educational and social policy. A growing body of methodological research sheds light on the validity and reliability of social network survey data regarding a single relation, but much less attention has been paid to the measurement of multiplex networks and the validity of comparisons among criterion relations. In this paper, we identify ways that surveys designed to collect multiplex social network data might be vulnerable to question-order effects. We then test several hypotheses using a split-ballot experiment embedded in an online multiple name generator survey of teachers’ advice networks, collected for a study of complete networks. We conclude by discussing implications for the design of multiple name generator social network surveys.

[1]  P. Killworth,et al.  INFORMANT ACCURACY IN SOCIAL NETWORK DATA II , 1977 .

[2]  Seymour Sudman,et al.  Thinking about Answers: The Application of Cognitive Process to Survey Methodology , 1996 .

[3]  Floyd J. Fowler,et al.  The Case for More Split‐Sample Experiments in Developing Survey Instruments , 2004 .

[4]  Nataša Golubovi,et al.  NETWORK STRUCTURE OF SOCIAL CAPITAL , 2010 .

[5]  T. Snijders,et al.  p2: a random effects model with covariates for directed graphs , 2004 .

[6]  S. Stodolsky.  The Subject Matters: Classroom Activity in Math and Social Studies , 1988 .

[7]  Alexandra Marin,et al.  Simplifying the Personal Network Name Generator , 2007 .

[8]  James P. Spillane,et al.  High-Stakes Accountability in Urban Elementary Schools: Challenging or Reproducing Inequality?1 , 2004, Teachers College Record: The Voice of Scholarship in Education.

[9]  Andrew Parker,et al.  Beyond answers: dimensions of the advice network , 2001, Soc. Networks.

[10]  C. Coburn,et al.  Collective Sensemaking about Reading: How Teachers Mediate Reading Policy in Their Professional Communities , 2001 .

[11]  P. Killworth,et al.  Informant Accuracy in Social Network Data , 1976 .

[12]  Thomas W. Valente,et al.  The stability of centrality measures when networks are sampled , 2003, Soc. Networks.

[13]  K. Frank,et al.  Social Capital and the Diffusion of Innovations Within Organizations: The Case of Computer Technology in Schools , 2004 .

[14]  L. Freeman,et al.  Cognitive Structure and Informant Accuracy , 1987 .

[15]  R. Burt.  A note on social capital and network content , 1997 .

[16]  Corey Drake,et al.  Storied identities: Teacher learning and subject-matter context , 2001 .

[17]  N. Schwarz.  Self-reports: How the questions shape the answers. , 1999 .

[18]  James N. Baron,et al.  Resources and Relationships: Social Networks and Mobility in the Workplace , 1997 .

[19]  S. Wasserman,et al.  Models and methods in social network analysis , 2005 .

[20]  N H Martin,et al.  Men and machines. , 1973, Journal of clinical pathology.

[21]  P. Killworth,et al.  Informant accuracy in social-network data V. An experimental attempt to predict actual communication from recall data☆ , 1982 .

[22]  C. McCarty,et al.  COMPARING FOUR DIFFERENT METHODS FOR MEASURING PERSONAL SOCIAL NETWORKS , 1990 .

[23]  N. Schwarz,et al.  Asking Questions About Behavior: Cognition, Communication, and Questionnaire Construction , 2001 .

[24]  J. Coleman,et al.  Social Capital in the Creation of Human Capital , 1988, American Journal of Sociology.

[25]  Roger Tourangeau,et al.  Cognition and survey research , 2000 .

[26]  Devon D. Brewer,et al.  Forgetting in the recall-based elicitation of personal and social networks , 2000, Soc. Networks.

[27]  Giovanni Rinaldi,et al.  A Quantitative Review of Associative Patterns in the Recall of Persons , 2005, J. Soc. Struct..

[28]  Bruce C. Straits,et al.  Ego's important discussants or significant people: an experiment in varying the wording of personal network name generators , 2000, Soc. Networks.

[29]  Marc T. Braverman,et al.  Advances in Survey Research. , 1996 .

[30]  Anuska Ferligoj,et al.  Estimating the reliability and validity of personal support measures: full information ML estimation with planned incomplete data , 2002, Soc. Networks.

[31]  R. Tourangeau,et al.  Cognitive Processes Underlying Context Effects in Attitude Measurement , 1988 .

[32]  P. V. Marsden,et al.  Models and Methods in Social Network Analysis: Recent Developments in Network Measurement , 2005 .

[33]  Matthew E. Brashears,et al.  Social Isolation in America: Changes in Core Discussion Networks over Two Decades , 2006 .

[34]  Charles D. Cowan,et al.  EFFECTS OF SUPPLEMENTAL QUESTIONS ON VICTIMIZATION ESTIMATES FROM THE NATIONAL CRIME SURVEY , 2002 .

[35]  C. Hardy.  Organizations: Rational, Natural and Open Systems , 1983 .

[36]  James P. Spillane.  Primary school leadership practice: how the subject matters1 , 2005 .

[37]  Jon A. Krosnick,et al.  Satisficing in surveys: Initial evidence , 1996 .

[38]  Valentina Hlebec,et al.  Reliability of measures of centrality and prominence , 2005, Soc. Networks.

[39]  Peter V. Marsden,et al.  Interpretation and interview context: examining the General Social Survey name generator using cognitive methods , 1999, Soc. Networks.

[40]  James P. Spillane,et al.  Understanding teacher leadership in middle school mathematics: A collaborative research effort , 2009 .

[41]  Danielle De Lange,et al.  Asking Social Network Questions: A Quality Assessment of Different Measures , 2004 .

[42]  C. Coburn,et al.  District Policy and Teachers’ Social Networks , 2008 .

[43]  B. Uzzi,et al.  Social Structure and Competition in Interfirm Networks: The Paradox of Embeddedness , 1997 .

[44]  P. Adler,et al.  Social Capital: Prospects for a New Concept , 2002 .

[45]  Steven B. Andrews,et al.  Structural Holes: The Social Structure of Competition , 1995, The SAGE Encyclopedia of Research Design.

[46]  Emmanuel Lazega,et al.  Multiplexity, generalized exchange and cooperation in organizations: a case study , 1999, Soc. Networks.

[47]  N. Schwarz,et al.  Thinking About Answers: The Application of Cognitive Processes to Survey Methodology , 1995, Quality of Life Research.

[48]  John MacBeath,et al.  Leadership for learning : international perspectives , 2008 .

[49]  John W. Meyer.  Institutional and technical sources of organizational structure explaining the structure of educational organizations , 1980 .

[50]  James P. Spillane,et al.  Professional Community or Communities? School Subject Matter and Elementary School Teachers’ Work Environments (WP-05-02) , 2005 .

[51]  Ali Haider,et al.  Partner naming and forgetting: Recall of network members , 2007, Soc. Networks.

[52]  James P. Spillane,et al.  How Subjects Matter in District Office Practice: Instructionally Relevant Policy in Urban School District Redesign , 2005 .

[53]  Mart G.M. van der Poel,et al.  Delineating personal support networks , 1993 .

[54]  P. Killworth,et al.  Informant accuracy in social network data IV: a comparison of clique-level structure in behavioral and cognitive network data , 1979 .

[55]  Elizabeth Martin,et al.  METHODS FOR TESTING AND EVALUATING SURVEY QUESTIONS , 2004 .

[56]  Ray Reagans,et al.  Network Structure and Knowledge Transfer: The Effects of Cohesion and Range , 2003 .

[57]  Danching Ruan,et al.  The content of the General Social Survey discussion networks : an exploration of General Social Survey discussion name generator in a Chinese context , 1998 .

[58]  Vasja Vehovar,et al.  Measuring ego-centered social networks on the web: Questionnaire design issues , 2008, Soc. Networks.

[59]  Seymour Sudman,et al.  Experiments in the measurement of the size of social networks , 1985 .

[60]  Brian Uzzi,et al.  Relational Embeddedness and Learning: The Case of Bank Loan Managers and Their Clients , 2003, Manag. Sci..

[61]  C. Coburn,et al.  Shaping Teacher Sensemaking: School Leaders and the Enactment of Reading Policy , 2005 .

[62]  M. Hammer,et al.  Explorations into the meaning of social network interview data , 1984 .

[63]  Alexandra Marin,et al.  Are respondents more likely to list alters with certain characteristics?: Implications for name generator data , 2004, Soc. Networks.

[64]  Barry Wellman,et al.  It's not who you know, it's how you know them: Who exchanges what with whom? , 2007, Soc. Networks.

[65]  Garry Robins,et al.  A spatial model for social networks , 2006 .

[66]  James P. Spillane.  Standards deviation: How local schools misunderstand policy , 2004 .

[67]  Anthony S. Bryk,et al.  Finding a Measure of Trust@@@Trust in Schools: A Core Resource for Improvement , 2004 .

[68]  Anuska Ferligoj,et al.  Evaluation of social network measurement instruments , 1999, Soc. Networks.