Quality Issues with Survey Research

This paper identifies some of the many decision‐making stages in the analysis of survey data and some of the ways in which results are affected by analysis decisions. It argues for greater transparency in reporting the details of both the data and the methods of analysis used. This not only enables replication but also allows a much more critical appraisal of the research than is usually possible. This is particularly important where research makes claims about causal processes that may influence policy decisions. Achieving this requires a greater willingness by some journal editors to allow details of data collection and analysis to be fully reported, for example by greater use of associated websites. Appendix 1 lists a proposed set of reporting criteria that could be adopted by editors.

[1]  C. A. Moser,et al.  Survey Methods in Social Investigation. , 1958 .

[2]  J. Neyman.  On the Two Different Aspects of the Representative Method: the Method of Stratified Sampling and the Method of Purposive Selection , 1934 .

[3]  B. Meuleman,et al.  Attitudes towards migration in Europe: a cross-cultural and contextual approach , 2005 .

[4]  N. Wermuth,et al.  Some statistical aspects of causality , 2001 .

[5]  S. Presser,et al.  Survey Questions: Handcrafting the Standardized Questionnaire. , 1988 .

[6]  M. Kenward,et al.  A comparison of multiple imputation and doubly robust estimation for analyses with missing data , 2006 .

[7]  J. Heckman.  Sample selection bias as a specification error , 1979 .

[8]  E. Ziegel,et al.  Nonresponse In Household Interview Surveys , 1998 .

[9]  Sue Duncan,et al.  A User Perspective on Research Quality , 2006 .

[10]  Matthew Salo.  Nonresponse in Household Interview Surveys , 1999 .

[11]  C. Manski.  Identification of Endogenous Social Effects: The Reflection Problem , 1993 .

[12]  Robert M. Groves,et al.  Nonresponse in Household Interview Surveys: Groves/Nonresponse , 1998 .

[13]  Lucy Dillon,et al.  Quality in qualitative evaluation: A framework for assessing research evidence. A quality framework. , 2003 .

[14]  A. Bryman.  Social Research Methods , 2001 .

[15]  A. Bryman.  Quantity and quality in social research , 1988 .

[16]  P G Gipps,et al.  Elements of sampling theory , 1984 .

[17]  Andrew J. Oswald,et al.  Unionism and Employment Behaviour , 1991 .

[18]  M. N. Bhrolcháin.  ‘Divorce Effects’ and Causality in the Social Sciences , 2001 .

[19]  Nicole A. Lazar,et al.  Statistical Analysis With Missing Data , 2003, Technometrics.

[20]  L. Simpson,et al.  Quantifying stability and change in ethnic group , 2007 .

[21]  D. Rubin,et al.  Multiple Imputation for Nonresponse in Surveys , 1989 .

[22]  M. Couper,et al.  Nonresponse in cross-cultural and cross-national surveys , 2003 .

[23]  M. Iacovou.  "Family Composition and Children's Educational Outcomes" , 2001 .