Sensitive Questions in Online Surveys: Experimental Results for the Randomized Response Technique (RRT) and the Unmatched Count Technique (UCT)

Gaining valid answers to so-called sensitive questions is an age-old problem in survey research. Various techniques have been developed to guarantee anonymity and minimize the respondent’s feelings of jeopardy. Two such techniques are the randomized response technique (RRT) and the unmatched count technique (UCT). In this study the authors evaluate the effectiveness of different implementations of the RRT (using a forced-response design) in a computer-assisted setting and also compare the use of the RRT to that of the UCT. The techniques are evaluated according to various quality criteria, such as the prevalence estimates they provide, the ease of their use, and respondent trust in the techniques. The results indicate that the RRTs are problematic with respect to several domains, such as the limited trust they inspire and nonresponse, and that the RRT estimates are unreliable due to a strong false no bias, especially for the more sensitive questions. The UCT, however, performed well compared to the RRTs on all the evaluated measures. The authors conclude that the UCT is a promising alternative to RRT in self-administered surveys and that future research should be directed toward evaluating and improving the technique.

[1]  George B. Macready,et al.  Respondents' perceived protection when using randomized response. , 1982 .

[2]  G. Tian,et al.  Two new models for survey sampling with sensitive characteristic: design and analysis , 2008 .

[3]  Edmund J. Malesky,et al.  Foreign Investment and Bribery: A Firm-Level Analysis of Corruption in Vietnam , 2012 .

[4]  Gerty J. L. M. Lensvelt-Mulders,et al.  Evaluating compliance with a computer assisted randomized response technique: a qualitative study into the origins of lying and cheating , 2007, Comput. Hum. Behav..

[5]  Thomas L. Martin,et al.  Academic Economists Behaving Badly? A Survey on Three Areas of Unethical Behavior , 2001 .

[6]  Johannes A. Landsheer,et al.  Trust and Understanding, Two Psychological Aspects of Randomized Response , 1999 .

[7]  Samuel Himmelfarb,et al.  Social desirability and the randomized response technique. , 1982 .

[8]  Thomas A. Buchman,et al.  Obtaining Responses To Sensitive Questions - Conventional Questionnaire Versus Randomized-Response Technique , 1982 .

[9]  Walter T. Federer,et al.  Block Total Response as an Alternative to the Randomized Response Method in Surveys , 1979 .

[10]  Robert A. Desharnais,et al.  Honest Answers to Embarrassing Questions: Detecting Cheating in the Randomized Response Model , 1998 .

[11]  Ivar Krumpal.  Determinants of social desirability bias in sensitive surveys: a literature review , 2013 .

[12]  Anthony Y. C. Kuk,et al.  Asking sensitive questions indirectly , 1990 .

[13]  Dan R. Dalton,et al.  USING THE UNMATCHED COUNT TECHNIQUE (UCT) TO ESTIMATE BASE RATES FOR SENSITIVE BEHAVIOR , 1994 .

[14]  Arijit Chaudhuri,et al.  Item Count Technique in estimating the proportion of people with a sensitive feature , 2007 .

[15]  C. Mitchell Dayton,et al.  Covariate Randomized Response Models , 1988 .

[16]  NADINE RECKER RAYBURN,et al.  Base Rates of Hate Crime Victimization among College Students , 2003, Journal of interpersonal violence.

[17]  S. Edgell,et al.  Validity of Forced Responses in a Randomized Response Model , 1982 .

[18]  Peter G. M. van der Heijden,et al.  A validation of a computer‐assisted randomized response survey to estimate the prevalence of fraud in social security , 2006 .

[19]  Wendy Visscher,et al.  The Item Count Technique as a Method of Indirect Questioning: A Review of Its Development and a Case Study Application , 2011 .

[20]  C. Mitchell Dayton,et al.  Improved estimation of academic cheating behavior using the randomized response technique , 1987 .

[21]  Ulf Böckenholt,et al.  Item Randomized-Response Models for Measuring Noncompliance: Risk-Return Perceptions, Social Influences, and Self-Protective Responses , 2007 .

[22]  Paul E. Tracy,et al.  The Validity of Randomized Response for Sensitive Measurements , 1981 .

[23]  Steven E. Markham,et al.  An Empirical Examination of the Multi-dimensionality of Ethical Climate in Organizations , 1997 .

[24]  Ardo van den Hout,et al.  Estimating the linear regression model with categorical covariates subject to randomized response , 2006, Comput. Stat. Data Anal..

[25]  Jodie Houston,et al.  A Survey of Tax Evasion Using the Randomized Response Technique , 2001, Contemporary Issues in Taxation Research.

[26]  Peter G. M. van der Heijden,et al.  Meta-Analysis of Randomized Response Research , 2005 .

[27]  S. Gosling,et al.  Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. , 2004, The American psychologist.

[28]  Bernard Wong-On-Wing,et al.  A comparative examination of auditor premature sign-offs using the direct and the randomized response methods , 1997 .

[29]  J. Strickler,et al.  Measuring Induced Abortion in Mexico , 2004 .

[30]  Ben Jann,et al.  Plagiarism in Student Papers: Prevalence Estimates Using Special Techniques for Sensitive Questions , 2011 .

[31]  Ulf-Dietrich Reips.  Standards for Internet-based experimenting. , 2002, Experimental psychology.

[32]  Jamshid C. Hosseini,et al.  An Empirical Comparison of Direct Questioning, Scenario, and Randomized Response Methods for Obtaining Sensitive Business Information* , 1991 .

[33]  Donald E. Stem,et al.  Telephone Interview and Mail Questionnaire Applications of the Randomized Response Model , 1984 .

[34]  Arijit Chaudhuri,et al.  Optional versus compulsory randomized response techniques in complex surveys , 2005 .

[35]  Takahiro Tsuchiya,et al.  A Study of the Properties of the Item Count Technique , 2007 .

[36]  K. Rasinski,et al.  Methods of data collection, perceptions of risks and losses, and motivation to give truthful answers to sensitive survey questions , 1999 .

[37]  Bernard G. Greenberg,et al.  Estimates of induced abortion in urban North Carolina , 1970, Demography.

[38]  S L Warner,et al.  Randomized response: a survey technique for eliminating evasive answer bias. , 1965, Journal of the American Statistical Association.

[39]  Paul R. Sackett,et al.  A New Method of Examining Relationships between Individual Difference Measures and Sensitive Behavior Criteria: Evaluating the Unmatched Count Technique , 2004 .

[40]  Eleanor Singer,et al.  CONFIDENTIALITY ASSURANCES AND RESPONSE A QUANTITATIVE REVIEW OF THE EXPERIMENTAL LITERATURE , 1995 .

[41]  Diana C. Robertson,et al.  Purchasing Agents’ Deceptive Behavior: A Randomized Response Technique Study , 2001, Business Ethics Quarterly.

[42]  Andrei Postoaca,et al.  The Anonymous Elect: Market Research Through Online Access Panels , 2005 .

[43]  Ernest R. Larkins,et al.  The Validity Of The Randomized Response Method In Tax Ethics Research , 2011 .

[44]  Jon A. Krosnick,et al.  Measuring Voter Turnout By Using The Randomized Response Technique , 2010 .

[45]  R. Tourangeau,et al.  Sensitive questions in surveys. , 2007, Psychological bulletin.

[46]  Isabelle N. Rhodes,et al.  Making the Randomized Response Technique Work , 1976 .

[47]  Michael S. Goodstadt,et al.  The Randomized Response Technique: A Test on Drug Use , 1975 .

[48]  Bernard G. Greenberg,et al.  The Two Alternate Questions Randomized Response Model for Human Surveys , 1973 .

[49]  W. R. Simmons,et al.  The Unrelated Question Randomized Response Model: Theoretical Framework , 1969 .

[50]  Charles James Nice Bailey,et al.  Research Misconduct in Accounting Literature: A Survey of the Most Prolific Researchers' Actions and Beliefs , 2001 .

[51]  G. Maddala.  Limited-dependent and qualitative variables in econometrics: Introduction , 1983 .

[52]  L. Rips,et al.  The Psychology of Survey Response , 2000 .

[53]  M. Earleywine,et al.  Effect of response format on endorsement of eating disordered attitudes and behaviors. , 2007, The International journal of eating disorders.

[54]  Jon A. Krosnick,et al.  Social desirability bias in voter turnout reports Tests using the item count technique , 2010 .

[55]  Arndt Bröder,et al.  Improving survey research on the World-Wide Web using the randomized response technique , 2001 .

[56]  Dan R. Dalton,et al.  Base rate for employee theft: Convergence of multiple methods. , 1997 .

[57]  M. Earleywine,et al.  Sexual risk behaviors and alcohol: Higher base rates revealed using the unmatched‐count technique , 2000 .

[58]  J. Hox,et al.  A Comparison of Randomized Response, Computer-Assisted Self-Interview, and Face-to-Face Direct Questioning , 2000 .

[59]  Julie Barnett,et al.  Sensitive questions and response effects: an evaluation , 1998 .

[60]  Ivar Krumpal,et al.  Asking sensitive questions: the impact of forgiving wording and question context on social desirability bias , 2012 .

[61]  Paul E. Tracy,et al.  Randomized Response: A Method for Sensitive Surveys , 1986 .

[62]  Robert A. Desharnais,et al.  Honest Answers to Embarrassing Questions: Detecting Cheating in the Randomized Response Model , 1998 .