Confirmation Bias in Housing Unit Listing

Field listing of housing units is an expensive and time-consuming stage of the survey process, and its error properties are not well understood. This paper uses an experimental repeated listing design to demonstrate the presence of confirmation bias in dependent listing. We find evidence that when provided with an initial listing to update in the field, listers can become too trusting of the list and tend not to add missing units or delete inappropriate units. This finding has implications not only for surveys that use dependent listing to create housing unit frames but also for studies of coverage of housing unit frames.1 1Many thanks to James Lepkowski, Ashley Bowers, Adam Schlecte, Andrew Hupp, Kat Donahue and Matt Jans for their assistance with this experiment, and to the listers for their participation. Thanks also to Roger Tourangeau, Carolina Casas-Cordero and Michael Lemay for comments on earlier drafts, and to Austin Nichols and Ben Jann for discussions of the analysis. We also acknowledge the helpful comments of the journal reviewers and the financial support for this work from the Rensis Likert Fund at the University of Michigan and the ETH in Zurich, Switzerland.

[1]  Irene Hess,et al.  On Noncoverage of Sample Dwellings , 1958 .

[2]  Paul P. Biemer,et al.  Introduction to Survey Quality , 2003 .

[3]  Vincent G. Iannacchione,et al.  Evaluating the Use of Residential Mailing Addresses in a Metropolitan Household Survey , 2003 .

[4]  Michael W. Link,et al.  An Evaluation of Respondent Selection Methods for Household Mail Surveys , 2008 .

[5]  C. Mood.  Logistic Regression: Why We Cannot Do What We Think We Can Do, and What We Can Do About It , 2010 .

[6]  Seymour Sudman,et al.  Nonsampling Error in Surveys , 1993 .

[7]  F. S. P. Szuster,et al.  Nonsampling Error in Surveys , 1994 .

[8]  Stephanie Eckman,et al.  Applied sampling for large-scale multistage area probability designs , 2009 .

[9]  R. Groves,et al.  Planning and development of the continuous National Survey of Family Growth. , 2009, Vital and health statistics. Ser. 1, Programs and collection procedures.

[10]  C. O'muircheartaigh.  Simple Response Variance: Estimation and Determinants , 2011 .

[11]  Kosuke Imai,et al.  Survey Sampling , 1998, Nov/Dec 2017.

[12]  C. Ai,et al.  Interaction terms in logit and probit models , 2003 .

[13]  Stephanie Eckman,et al.  TRADITIONAL AND ENHANCED FIELD LISTING FOR PROBABILITY SAMPLING , 2002 .

[14]  Michael W. Link,et al.  A Comparison of Address-Based Sampling (ABS) Versus Random-Digit Dialing (RDD) for General Population Surveys , 2008 .

[15]  Dean Manheimer,et al.  Interviewer Performance in Area Sampling , 1949 .

[16]  J. Klayman.  Varieties of Confirmation Bias , 1995 .

[17]  J. Gill.  Hierarchical Linear Models , 2005 .

[18]  J. Lepkowski,et al.  Validating a Sampling Revolution : Benchmarking Address Lists against Traditional Listing , 2006 .

[19]  M. H. Hansen,et al.  Control of Errors in Surveys , 1956 .

[20]  J. Kmenta.  Mostly Harmless Econometrics: An Empiricist's Companion , 2010 .

[21]  Tommy Wright,et al.  A FRAME ON FRAMES: AN ANNOTATED BIBLIOGRAPHY , 1983 .

[22]  Tx Station.  Stata Statistical Software: Release 7. , 2001 .

[23]  R. Groves,et al.  Survey Errors and Survey Costs. , 1991 .

[24]  Vincent G. Iannacchione,et al.  EVALUATING THE USE OF RESIDENTIAL MAILING ADDRESSES IN A NATIONAL HOUSEHOLD SURVEY , 2003 .

[25]  Jeffrey M. Woodbridge.  Econometric Analysis of Cross Section and Panel Data , 2002 .

[26]  Jill M. Montaquila,et al.  Using a “Match Rate” Model to Predict Areas Where USPS-Based Address Lists May Be Used in Place of Traditional Listing , 2011 .

[27]  Predicting the Relative Quality of Alternative Sampling Frames , 2007 .

[28]  Peter Lynn,et al.  MEASURING CHANGE IN EMPLOYMENT CHARACTERISTICS: THE EFFECTS OF DEPENDENT INTERVIEWING , 2005 .