A Response Propensity Modeling Navigator for Paradata

The purpose of this paper is to identify factors affecting nonresponse of 12th graders in the National Assessment of Educational Progress (NAEP), by using social isolation as a theoretical navigator. In this paper, we also evaluate the statistical impact of nonresponse bias on estimates of educational performance in NAEP by taking advantage of response propensity models built on a social isolation framework. We use the 2000 NAEP science survey data and its contact history paradata, both of which are linked to the school administrative data from over 20,000 seniors in the 2000 High School Transcript Study (HSTS) whose sampling frame is identical to NAEP. We apply the final robust response propensity model to reweight NAEP estimates with additional covariates extracted from the HSTS administrative data. We evaluate the re-weighted Science performance estimates by comparing with those obtained using the current approach of NAEP nonresponse adjustment which relies on a few sampling frame variables just from NAEP data. Findings support recent research showing minimal effects on nonresponse bias of low response rates. We introduce the concept of "pandata," the data linked among multiple sources including administrative data and paradata, used for improving nonresponse adjustment methods to correct for potential nonresponse bias in survey research.

[1]  Robert Perkins,et al.  The 2005 High School Transcript Study User's Guide and Technical Report. NCES 2009-480. , 2008 .

[2]  A. Mcevoy,et al.  Antisocial Behavior, Academic Failure, and School Climate , 2000 .

[3]  Stanley Presser,et al.  How social processes distort measurement: the impact of survey nonresponse on estimates of volunteer work in the United States. , 2008, AJS; American journal of sociology.

[4]  R. Groves,et al.  Survey Errors and Survey Costs. , 1991 .

[5]  Roger Tourangeau,et al.  Experiments in Producing Nonresponse Bias , 2006 .

[6]  Robert J. Mislevy,et al.  BILOG 3 : item analysis and test scoring with binary logistic models , 1990 .

[7]  E. Skinner,et al.  Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. , 1993 .

[8]  J. Goyder,et al.  The Silent Minority: Nonrespondents on Sample Surveys. , 1989 .

[9]  Tom W. Smith.  The Hidden 25 Percent: An Analysis of Nonresponse on the 1980 General Social Survey , 1983 .

[10]  J. House,et al.  The association of social relationships and activities with mortality: prospective evidence from the Tecumseh Community Health Study. , 1982, American journal of epidemiology.

[11]  L. Meeuwesen,et al.  Social Isolation in Modern Society , 2006 .

[12]  Robert M. Groves,et al.  The Impact of Nonresponse Rates on Nonresponse Bias A Meta-Analysis , 2008 .

[13]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[14]  R M Groves,et al.  Consequences of reducing nonresponse in a national telephone survey. , 2000, Public opinion quarterly.

[15]  E. Singer,et al.  The effects of response rate changes on the index of consumer sentiment. , 2000, Public opinion quarterly.

[16]  E. Ziegel,et al.  Nonresponse In Household Interview Surveys , 1998 .

[17]  S. Smoyak.  LONELINESS: A SOURCEBOOK OF CURRENT THEORY, RESEARCH AND THERAPY , 1984 .

[18]  Robert Perkins,et al.  The High School Transcript Study: The 2000 High School Transcript Study User's Guide and Technical Report. NCES 2005-483. , 2005 .

[19]  R. Groves.  Nonresponse Rates and Nonresponse Bias in Household Surveys , 2006 .

[20]  Katharine G. Abraham,et al.  Non-Response in the American Time Use Survey: Who is Missing from the Data and How Much Does it Matter? , 2006 .

[21]  S. Neuman,et al.  Toward a Collaborative Approach to Parent Involvement in Early Education: A Study of Teenage Mothers in an African-American Community , 1995 .