Reliability and Validity of Measuring Social Support Networks by Web and Telephone

Egocentered networks are common in social science r esearch. Here, the unit of analysis is a respondent (ego) together wit h his/her personal network (alters). Usually, several variables are measured t o describe the relationship between egos and alters. In this paper, the aim is to estimate the reliability and validity of the averages of these measures by t he multitrait-multimethod (MTMM) approach. In the study, web and telephone modes of data collection are compared on a convenience sample of 238 second year students at the Faculty of Social Sciences at the U niversity of Ljubljana. The data was collected in 2003. The results show th at the telephone mode produces more reliable data than the web mode of da ta collection. Also, method order effect was shown: the data collection mode used first produces data of lower reliability than the mode us ed for the second measurement. There were no large differences in val idity of measurement.

[1]  H. Cantril,et al.  Gauging Public Opinion. , 1944 .

[2]  D. Alwin.  Approaches to the Interpretation of Relationships in the Multitrait-Multimethod Matrix , 1973 .

[3]  J. Conley.  Asking questions: A practical guide to questionnaire design , 1983 .

[4]  Willem E. Saris,et al.  Memory effects in MTMM studies , 1995 .

[5]  P. Killworth,et al.  Informant accuracy in social network data III: A comparison of triadic structure in behavioral and cognitive data , 1979 .

[6]  P. Killworth,et al.  INFORMANT ACCURACY IN SOCIAL NETWORK DATA II , 1977 .

[7]  H. Marsh.  Confirmatory Factor Analyses of Multitrait-Multimethod Data: Many Problems and a Few Solutions , 1989 .

[8]  Seymour Sudman,et al.  Autobiographical memory and the validity of retrospective reports , 1995 .

[9]  Anuska Ferligoj,et al.  Evaluation of social network measurement instruments , 1999, Soc. Networks.

[10]  Linton C. Freeman,et al.  Words, Deeds and Social Structure: A Preliminary Study of the Reliability of Informants , 1987 .

[11]  M. Browne,et al.  The decomposition of multitrait-multimethod matrices. , 1984, The British journal of mathematical and statistical psychology.

[12]  Seymour Sudman,et al.  Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research , 1995 .

[13]  N. Schwarz,et al.  Context Effects in Social and Psychological Research , 1992 .

[14]  Lluís Coromina,et al.  Reliability and validity of egocentered network data collected via web: A meta-analysis of multilevel multitrait multimethod studies , 2006, Soc. Networks.

[15]  A. C. Scherpenzeel.  A Question of Quality. Evaluating survey questions by multitrait-multimethod studies , 1995 .

[16]  A. Kimball Romney,et al.  Predicting informant accuracy from patterns of recall among individuals , 1984 .

[17]  Karen Cortell Reisman.  "The art of asking questions". , 2008, Texas dental journal.

[18]  J. Krosnick.  Response strategies for coping with the cognitive demands of attitude measures in surveys , 1991 .

[19]  Valentina Hlebec.  Reliability of Network Measurements , 1995 .

[20]  Anuska Ferligoj,et al.  Estimating the reliability and validity of personal support measures: full information ML estimation with planned incomplete data , 2002, Soc. Networks.

[21]  Willem E. Saris,et al.  8. A New Approach to Evaluating the Quality of Measurement Instruments: The Split-Ballot MTMM Design , 2004 .

[22]  N. Schwarz,et al.  Thinking About Answers: The Application of Cognitive Processes to Survey Methodology , 1995, Quality of Life Research.

[23]  Vasja Vehovar,et al.  The social support networks of internet users , 2006, New Media Soc..

[24]  Seymour Sudman,et al.  Thinking about Answers: The Application of Cognitive Process to Survey Methodology , 1996 .

[25]  A. Vaux.  Social Support: Theory, Research, and Intervention , 1988 .

[26]  Willem E. Saris,et al.  Testing Nested Additive, Multiplicative, and General Multitrait-Multimethod Models , 2000 .

[27]  L. Freeman,et al.  Cognitive Structure and Informant Accuracy , 1987 .

[28]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[29]  Vasja Vehovar,et al.  Collecting ego-centred network data via the web , 2004, Advances in Methodology and Statistics.

[30]  Anuska Ferligoj,et al.  Respondent mood and the instability of survey network measurements , 2001, Soc. Networks.

[31]  P. Killworth,et al.  Informant Accuracy in Social Network Data , 1976 .

[32]  R. P. McDonald,et al.  Evaluation of Measurement Instruments by Meta-Analysis of Multitrait Multimethod Studies. , 1992 .

[33]  Katherine Faust,et al.  Predicting the structure of a communications network from recalled data , 1982 .

[34]  Robert L. Linn,et al.  Path analysis: Psychological examples. , 1970 .

[35]  Anuska Ferligoj,et al.  Effects on reliability and validity of egocentered network measurements , 2005, Soc. Networks.