Effects of Answer Space Size on Responses to Open-ended Questions in Mail Surveys

Open-ended questions can provide valuable information to help researchers understand a respondent’s thinking but using such questions has proven problematic in mail surveys. Experimental data are used to assess the impact of answer space size on the length and content of responses to two open-ended questions that are part of Florida Cooperative Extension’s customer satisfaction survey. Based on 2,200 responses collected from 2003 to 2006, the size of the answer space showed no effect on the propensity to answer, but did affect the length of the response, with larger spaces eliciting more words and themes. In addition, larger spaces were more likely to have details elaborated. These results suggest that it is better to design larger answer spaces to more fully capture high-quality responses.

[1]  M Schemper,et al.  Explained variation for logistic regression. , 1996, Statistics in medicine.

[2]  Jolene D. Smyth,et al.  Helping Respondents Get It Right the First Time: The Influence of Words, Symbols, and Graphics in Web Surveys , 2007 .

[3]  S. Presser,et al.  Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context , 1996 .

[4]  Jolene D. Smyth,et al.  Open-Ended Questions in Web Surveys Can Increasing the Size of Answer Boxes and Providing Extra Verbal Instructions Improve Response Quality? , 2009 .

[5]  Jolene D. Smyth,et al.  How Visual Grouping Influences Answers to Internet Surveys 1 , 2004 .

[6]  J. Krosnick,et al.  Survey research. , 1999, Annual review of psychology.

[7]  J. A. Calvin.  Regression Models for Categorical and Limited Dependent Variables , 1998 .

[8]  F. Conrad,et al.  Spacing, Position, and Order Interpretive Heuristics for Visual Features of Survey Questions , 2004 .

[9]  F. Windmeijer,et al.  R-Squared Measures for Count Data Regression Models With Applications to Health-Care Utilization , 1996 .

[10]  Roger Tourangeau,et al.  Survey research and societal change. , 2004, Annual review of psychology.

[11]  F. Famoye,et al.  Modeling household fertility decisions with generalized Poisson regression , 1997, Journal of population economics.

[12]  D. A. Dillman,et al.  Achieving Usability in Establishment Surveys Through the Application of Visual Design Principles , 2005 .

[13]  N. C. Schaeffer,et al.  The Science of Asking Questions , 2003 .

[14]  Cleo D. Redline,et al.  Improving Navigational Performance in U.S. Census 2000 by Altering the Visually Administered Languages of Branching Instructions , 2003 .

[15]  Edith D. de Leeuw,et al.  Choosing the Method of Data Collection , 2008 .

[16]  M Schemper,et al.  Computing measures of explained variation for logistic regression models. , 1999, Computer methods and programs in biomedicine.

[17]  Don A. Dillman,et al.  The Influence of Graphical and Symbolic Language Manipulations on Responses to Self-Administered Questions , 2004 .

[18]  P. Chisnall.  Mail and Internet Surveys: The Tailored Design Method , 2007, Journal of Advertising Research.

[19]  Shirley A. Star,et al.  AMERICAN ASSOCIATION FOR PUBLIC OPINION RESEARCH , 1980 .

[20]  Jolene D Smyth,et al.  Visual Design, Order Effects, and Respondent Characteristics in a Self-Administered Survey , 2007 .

[21]  R. Tourangeau,et al.  Cognitive Processes Underlying Context Effects in Attitude Measurement , 1988 .

[22]  D. Dillman.  Mail and internet surveys: The tailored design method, 2nd ed. , 2007 .

[23]  Martina Mittlböck,et al.  Adjustments for R 2 -measures for Poisson regression models , 2000 .

[24]  F. Strack,et al.  A comparison of response effects in self-administered and telephone surveys , 1987 .

[25]  Jon A. Krosnick,et al.  Satisficing in surveys: Initial evidence , 1996 .

[26]  E. Mulvey,et al.  Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models. , 1995, Psychological bulletin.