Tracking the Source of Measurement Error

Between 2008 and 2012, 591 students filled out a questionnaire in which they evaluated probability statements. This questionnaire was designed to examine the source of error. Errors occurred in 60.9% of the questionnaires. For the most part, the anticipated sources of the errors were not significant: length of the instrument, the promise of extra credit or the order of the questions. GPA had a limited effect. The largest effect was that of gender.

[1]  Mark C. Long,et al.  Why Are Men Falling Behind? Gender Gaps in College Performance and Persistence , 2010 .

[2]  K. Cokley,et al.  Moderating and Mediating Effects of Gender and Psychological Disengagement on the Academic Achievement of African American College Students , 2007 .

[3]  M. Biehl,et al.  Adolescents' and adults' understanding of probability expressions. , 2001, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[4]  Robert M. Groves,et al.  Total Survey Error: Past, Present, and Future , 2010 .

[5]  Robert M. Groves,et al.  RESEARCH ON SURVEY DATA QUALITY , 1987 .

[6]  Tom W. Smith,et al.  Motivation to Report Sensitive Behaviors on Surveys: Evidence From a Bogus Pipeline Experiment1 , 1997 .

[7]  Matthew N. Beckmann,et al.  What Leads to Voting Overreports? Contrasts of Overreporters to Validated Voters and Admitted Nonvoters in the American National Election Studies , 2001 .

[8]  E. Singer,et al.  The effects of response rate changes on the index of consumer sentiment. , 2000, Public opinion quarterly.

[9]  Keiko Nakao,et al.  THE 1989 SOCIOECONOMIC INDEX OF OCCUPATIONS: CONSTRUCTION FROM THE 1989 OCCUPATIONAL PRESTIGE SCORES , 1989 .

[10]  E. Singer,et al.  Experiments with incentives in telephone surveys. , 2000, Public opinion quarterly.

[11]  N. Bradburn,et al.  An Investigation of Interview Method, Threat and Response Distortion , 1976 .

[12]  C. K. Mertz,et al.  Gender, race, and perceived risk: The 'white male' effect , 2000 .

[13]  R. Simon,et al.  Quantifying Burdens of Proof: A View from the Bench, the Jury, and the Classroom , 1971 .

[14]  Fritz Drasgow,et al.  A Meta-Analytic Study of Social Desirability Distortion in Computer- Administered Questionnaires, Traditional Questionnaires, and Interviews , 1999 .

[15]  Ross A. Thompson,et al.  Extra Credit as Incentive for Voluntary Research Participation , 2005 .

[16]  Paul P. Biemer,et al.  Introduction to Survey Quality , 2003 .

[17]  Stephen R. Porter,et al.  Non-response in student surveys: The Role of Demographics, Engagement and Personality , 2005 .

[18]  Amy M. Hightower,et al.  Science and Engineering Indicators , 1993 .

[19]  N. Schwarz.  Self-reports: How the questions shape the answers. , 1999 .

[20]  Lei Lai,et al.  Social incentives for gender differences in the propensity to initiate negotiations: Sometimes it does hurt to ask , 2007 .

[21]  S. Mullainathan,et al.  Do People Mean What They Say? Implications for Subjective Survey Data , 2001 .

[22]  Walter C. Buboltz,et al.  Psychological reactance: examination across age, ethnicity, and gender. , 2007, The American journal of psychology.

[23]  Melanie C. Green,et al.  Telephone versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires: Comparisons of Respondent Satisficing and Social Desirability Response Bias , 2003 .

[24]  D. Dillman,et al.  EFFECTS OF QUESTIONNAIRE LENGTH, RESPONDENT-FRIENDLY DESIGN, AND A DIFFICULT QUESTION ON RESPONSE RATES FOR OCCUPANT-ADDRESSED CENSUS MAIL SURVEYS , 1993 .

[25]  T. Gale,et al.  Do mental health professionals really understand probability? Implications for risk assessment and evidence-based practice , 2003 .