Non-response in student surveys: The Role of Demographics, Engagement and Personality

What causes a student to participate in a survey? This paper looks at participation across multiple surveys to understand survey non-response; by using multiple surveys we minimize the impact of survey salience. Students at a selective liberal arts college were administered four different surveys throughout the 2002–2003 academic year, and we use the number of surveys participated in to understand how student characteristics such as demographics, engagement and Holland personality type affect cooperation. We find that survey respondents are more likely to be female and socially engaged, less likely to be on financial aid, more likely to be an investigative personality type and less likely to be an enterprising personality type.

[1]  T. C. Mccormick,et al.  Some Survey Sampling Biases , 1955 .

[2]  John L. Holland,et al.  The Psychology Of Vocational Choice , 1966 .

[3]  E. Donnerstein,et al.  Some implications for psychological research of early versus late term participation by college subjects , 1974 .

[4]  T. Heberlein,et al.  Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. , 1978 .

[5]  Shirley A. Star,et al.  AMERICAN ASSOCIATION FOR PUBLIC OPINION RESEARCH , 1980 .

[6]  C. Steeh.  Trends in Nonresponse Rates, 1952–1979 , 1981 .

[7]  Paul B. Sheatsley,et al.  Nonresponse Bias for Attitude Questions , 1981 .

[8]  J. Goyder,et al.  Further Evidence on Factors Affecting Response Rates to Mailed Questionnaires , 1982 .

[9]  L. Adams,et al.  Solving the quandary between questionnaire length and response rate in educational research , 1982 .

[10]  D. Kandel,et al.  Sex Differences in the Characteristics of Members Lost to a Longitudinal Panel: A Speculative Research Note , 1983 .

[11]  J. L. Holland,et al.  Making vocational choices : a theory of vocational personalities and work environments , 1984 .

[12]  J. Goyder,et al.  Surveys on Surveys: Limitations and Potentialities , 1986 .

[13]  Nona Tollefson,et al.  Response bias in college freshmen's responses to mail surveys , 1987 .

[14]  Paul B. Duby,et al.  An Evaluation of the Use of Monetary Incentives in Postsecondary Survey Research. , 1987 .

[15]  J. Goyder.  The Silent Minority , 2019 .

[16]  Trudy H. Bers,et al.  Improving alumni survey response rates: An experiment and cost-benefit analysis , 1987 .

[17]  R. Andersen,et al.  The Silent Minority: Nonrespondents on Sample Surveys.By John Goyder. Boulder, Colo.: Westview Press. 232 pp. $39.00 , 1990 .

[18]  H. Cooper,et al.  Do Students with Different Characteristics Take Part in Psychology Experiments at Different Times of the Semester , 1991 .

[19]  Robert M. Groves,et al.  UNDERSTANDING THE DECISION TO PARTICIPATE IN A SURVEY , 1992 .

[20]  J. Hoek,et al.  THE BEHAVIOR OF RESPONDENTS, NONRESPONDENTS, AND REFUSERS ACROSS MAIL SURVEYS , 1992 .

[21]  H. J. Liberty.  The relationship between extraversion and time of data collection , 1993 .

[22]  E. Borgida,et al.  Survey topic involvement and nonresponse bias , 1993 .

[23]  H. F. Koenig,et al.  MAIL SURVEY RESPONSE RATE PATTERNS IN A POPULATION OF THE ELDERLY: DOES RESPONSE DETERIORATE WITH AGE? , 1994 .

[24]  G. Moskowitz,et al.  Individual Differences In Experiment Participation: Structure, Autonomy, and the Time of the Semeste , 1995 .

[25]  Tom W. Smith.  TRENDS IN NON-RESPONSE RATES , 1995 .

[26]  Eleanor Singer,et al.  CONFIDENTIALITY ASSURANCES AND RESPONSE A QUANTITATIVE REVIEW OF THE EXPERIMENTAL LITERATURE , 1995 .

[27]  N. C. Schaeffer,et al.  Institute for Research on Poverty Discussion Paper no. 1024-93 Using Survey Participants to Estimate the Impact of Nonparticipation , 1993 .

[28]  Eric L. Dey.  Working with Low Survey Response Rates: The Efficacy of Weighting Adjustments , 1995 .

[29]  P. Costa,et al.  Toward a new generation of personality theories: Theoretical contexts for the five-factor model. , 1996 .

[30]  M. Couper.  Survey introductions and data quality , 1997 .

[31]  J. A. Calvin.  Regression Models for Categorical and Limited Dependent Variables , 1998 .

[32]  E. Singer,et al.  Differential Incentives: Beliefs about Practices, Perceptions of Equity, and Effects on Survey Participation , 1999 .

[33]  Heather Laurie,et al.  Strategies for reducing nonresponse in a longitudinal panel survey , 1999 .

[34]  Curt J. Dommeyer,et al.  Comparing Two Forms of an E-mail Survey: Embedded vs Attached , 2000 .

[35]  E. Singer,et al.  The effects of response rate changes on the index of consumer sentiment. , 2000, Public opinion quarterly.

[36]  R. Reardon,et al.  Academic Disciplines: Holland's Theory and the Study of College Students and Faculty (review) , 2000 .

[37]  R. Mowrer,et al.  Mail Surveys among College Students: Do Initial Responders Score Differently from Nonresponders on the 16PF? , 2000, Psychological reports.

[38]  L. Sax,et al.  The American Freshman: National Norms for Fall 2000. , 2000 .

[39]  E. Singer,et al.  Experiments with incentives in telephone surveys. , 2000, Public opinion quarterly.

[40]  E. Singer,et al.  Leverage-saliency theory of survey participation: description and an illustration. , 2000, Public opinion quarterly.

[41]  Paul D. Umbach,et al.  How Do Academic Departments Impact Student Satisfaction? Understanding the Contextual Effects of Departments , 2001 .

[42]  R M Weiler,et al.  The Feasibility of a Web-Based Surveillance System to Collect Health Risk Behavior Data from College Students , 2001, Health education & behavior : the official publication of the Society for Public Health Education.

[43]  Ronald A. Ash,et al.  The Conscientiousness of Students in Subject Pools: Implications for “Laboratory” Research , 2001 .

[44]  Paul D. Umbach,et al.  What Works Best? Collecting Alumni Data with Multiple Technologies. AIR 2001 Annual Forum Paper. , 2001 .

[45]  M. Couper,et al.  Web Surveys , 2001 .

[46]  M. Traugott,et al.  Web survey design and administration. , 2001, Public opinion quarterly.

[47]  P. H. Lodhi,et al.  The Five-Factor Model of Personality , 2002 .

[48]  Peter H. Siegel,et al.  National Postsecondary Student Aid Study 1999-2000 (NPSAS:2000), CATI Nonresponse Bias Analysis Report. Working Paper Series. , 2002 .

[49]  John M. Zelenski,et al.  Who comes when: personality differences in early and later participation in a university subject pool , 2002 .

[50]  E. D. de Leeuw,et al.  Trends in household survey nonresponse: a longitudinal and international comparison , 2002 .

[51]  K. Warriner,et al.  Evaluating Socio-economic Status (SES) Bias in Survey Nonresponse , 2002 .

[52]  Patrick Van Kenhove,et al.  The influence of topic involvement on mail survey response behavior , 2002 .

[53]  Peter Tuckel,et al.  The Vanishing Respondent In Telephone Surveys , 2002, Journal of Advertising Research.

[54]  Lisa M. Larson,et al.  Meta-analyses of Big Six Interests and Big Five Personality Factors , 2002 .

[55]  T. Koepsell,et al.  Characteristics of telephone survey respondents according to willingness to participate. , 2003, American journal of epidemiology.

[56]  Shannon K. Gilmartin,et al.  Assessing Response Rates and Nonresponse Bias in Web and Paper Surveys , 2003 .

[57]  Susan A. Ariew,et al.  National Survey of Student Engagement , 2003 .

[58]  Dike van de Mheen,et al.  Estimating non-response bias in a survey on alcohol consumption: comparison of response waves. , 2003, Alcohol and alcoholism.

[59]  Stephen R. Porter,et al.  The Impact of Lottery Incentives on Student Survey Response Rates , 2003 .

[60]  Murray R. Barrick,et al.  META‐ANALYSIS OF THE RELATIONSHIP BETWEEN THE FIVE‐FACTOR MODEL OF PERSONALITY AND HOLLAND'S OCCUPATIONAL TYPES , 2003 .

[61]  Stephen R. Porter,et al.  The Impact of Contact Type on Web Survey Response Rates , 2003 .

[62]  Michael E. Whitcomb,et al.  E-Mail Contacts , 2004 .

[63]  Stephen R. Porter,et al.  Multiple Surveys of Students and Survey Fatigue. , 2004 .

[64]  Paul D. Umbach,et al.  What Works Best? Collecting Alumni Data with Multiple Technologies Professional File Association for Institutional Research Number 90, Winter, 2004 Enhancing knowledge. Expanding networks. Professional Development, Informational Resources & Networking , 2004 .

[65]  John C. Weidman.  Academic Disciplines: Holland's Theory and the Study of College Students and Faculty , 2005 .

[66]  Paul D. Umbach,et al.  COLLEGE MAJOR CHOICE: An Analysis of Person–Environment Fit , 2006 .