Evaluation of Respondent-driven Sampling

Background: Respondent-driven sampling is a novel variant of link-tracing sampling for estimating the characteristics of hard-to-reach groups, such as HIV prevalence in sex workers. Despite its use by leading health organizations, the performance of this method in realistic situations is still largely unknown. We evaluated respondent-driven sampling by comparing estimates from a respondent-driven sampling survey with total population data. Methods: Total population data on age, tribe, religion, socioeconomic status, sexual activity, and HIV status were available on a population of 2402 male household heads from an open cohort in rural Uganda. A respondent-driven sampling (RDS) survey was carried out in this population, using current methods of sampling (RDS sample) and statistical inference (RDS estimates). Analyses were carried out for the full RDS sample and then repeated for the first 250 recruits (small sample). Results: We recruited 927 household heads. Full and small RDS samples were largely representative of the total population, but both samples underrepresented men who were younger, of higher socioeconomic status, and with unknown sexual activity and HIV status. Respondent-driven sampling statistical inference methods failed to reduce these biases. Only 31%–37% (depending on method and sample size) of RDS estimates were closer to the true population proportions than the RDS sample proportions. Only 50%–74% of respondent-driven sampling bootstrap 95% confidence intervals included the population proportion. Conclusions: Respondent-driven sampling produced a generally representative sample of this well-connected nonhidden population. However, current respondent-driven sampling inference methods failed to reduce bias when it occurred. Whether the data required to remove bias and measure precision can be collected in a respondent-driven sampling survey is unresolved. Respondent-driven sampling should be regarded as a (potentially superior) form of convenience sampling method, and caution is required when interpreting findings based on the sampling method.

[1]  Erik M. Volz,et al.  Probability based estimation theory for respondent driven sampling , 2008 .

[2]  Mark S Handcock,et al.  7. Respondent-Driven Sampling: An Assessment of Current Methodology , 2009, Sociological methodology.

[3]  Matthew J. Salganik.  Variance Estimation, Design Effects, and Sample Size Calculations for Respondent-Driven Sampling , 2006, Journal of Urban Health.

[4]  Rebeca Ramos,et al.  Respondent-Driven Sampling of Injection Drug Users in Two U.S.–Mexico Border Cities: Recruitment Dynamics and Impact on Estimates of HIV and Syphilis Prevalence , 2006, Journal of Urban Health.

[5]  Douglas D. Heckathorn,et al.  Effectiveness of Respondent-Driven Sampling for Recruiting Drug Users in New York City: Findings from a Pilot Study , 2006, Journal of Urban Health.

[6]  Douglas D. Heckathorn,et al.  Respondent-driven sampling : A new approach to the study of hidden populations , 1997 .

[7]  Matthew J. Salganik,et al.  5. Sampling and Estimation in Hidden Populations Using Respondent-Driven Sampling , 2004 .

[8]  Tobi Saidel,et al.  Review of sampling hard-to-reach and hidden populations for HIV surveillance. , 2005, AIDS.

[9]  Matthew J. Salganik,et al.  How Many People Do You Know?: Efficiently Estimating Personal Network Size , 2010, Journal of the American Statistical Association.

[10]  Cyprian Wejnert,et al.  3. An Empirical Test of Respondent-Driven Sampling: Point Estimates, Variance, Degree Measures, and Out-of-Equilibrium Data , 2009, Sociological methodology.

[11]  Lisa G. Johnston,et al.  An Empirical Comparison of Respondent-driven Sampling, Time Location Sampling, and Snowball Sampling for Behavioral Surveillance in Men Who Have Sex with Men, Fortaleza, Brazil , 2008, AIDS and Behavior.

[12]  Emden R. Gansner,et al.  An open graph visualization system and its applications to software engineering , 2000, Softw. Pract. Exp..

[13]  D. Heckathorn.  6. Extensions of Respondent-Driven Sampling: Analyzing Continuous Variables and Controlling for Differential Recruitment , 2007 .

[14]  Richard G. White,et al.  Evaluation of the role of location and distance in recruitment in respondent-driven sampling , 2011, International journal of health geographics.

[15]  Lisa G. Johnston,et al.  Methods to Recruit Hard-to-Reach Groups: Comparing Two Chain Referral Sampling Methods of Recruiting Injecting Drug Users Across Nine Studies in Russia and Estonia , 2006, Journal of Urban Health.

[16]  P. Kaye.  Infectious diseases of humans: Dynamics and control , 1993 .

[17]  Douglas D. Heckathorn,et al.  From Networks to Populations: The Development and Application of Respondent-Driven Sampling Among IDUs and Latino Gay Men , 2005, AIDS and Behavior.

[18]  A. Kamali,et al.  Seven-year trends in HIV-1 infection rates, and changes in sexual behaviour, among adults in rural Uganda , 2000, AIDS.

[19]  Min Xu,et al.  Trends in Prevalence of HIV, Syphilis, Hepatitis C, Hepatitis B, and Sexual Risk Behavior Among Men Who Have Sex With Men: Results of 3 Consecutive Respondent-Driven Sampling Surveys in Beijing, 2004 Through 2006 , 2007, Journal of acquired immune deficiency syndromes.

[20]  Matthew J. Salganik,et al.  Assessing respondent-driven sampling , 2010, Proceedings of the National Academy of Sciences.

[21]  Douglas D. Heckathorn,et al.  Respondent-driven sampling II: deriving valid population estimates from chain-referral samples of hi , 2002 .

[22]  L. Johnston,et al.  Efficacy of convenience sampling through the internet versus respondent driven sampling among males who have sex with males in Tallinn and Harju County, Estonia: challenges reaching a hidden population , 2009, AIDS care.

[23]  Mohsen Malekinejad,et al.  Using Respondent-Driven Sampling Methodology for HIV Biological and Behavioral Surveillance in International Settings: A Systematic Review , 2008, AIDS and Behavior.

[24]  C. McCarty,et al.  Comparing Two Methods for Estimating Network Size , 2001 .

[25]  Cyprian Wejnert,et al.  Web-Based Network Sampling , 2008 .

[26]  J. Sterne,et al.  Essential Medical Statistics , 2003 .

[27]  A. Kamali,et al.  HIV prevalence and incidence are no longer falling in southwest Uganda: evidence from a rural population cohort 1989–2005 , 2008, AIDS.

[28]  Holly Hagan,et al.  Evaluating respondent-driven sampling in a major metropolitan area: Comparing injection drug users in the 2005 Seattle area national HIV behavioral surveillance system survey with participants in the RAVEN and Kiwi studies. , 2010, Annals of epidemiology.

[29]  Stephanie Tortu,et al.  Recruiting Injection Drug Users: A Three-Site Comparison of Results and Experiences with Respondent-Driven and Targeted Sampling Procedures , 2006, Journal of Urban Health.