Test Procedure Bias: A Meta-Analysis of Examiner Familiarity Effects

This article presents a meta-analysis of the effects of examiner familiarity on children’s test performance. The data for the meta-analysis came from 22 controlled studies involving 1,489 subjects. In the typical study, the effect of examiner familiarity raised test performance by .28 standard deviations. Differential performance favoring the familiar examiner condition was greater when subjects were of low socioeconomic status (SES), were tested on comparatively difficult tests, and knew the examiner for a relatively long duration. Implications for theory and practice are discussed.

[1]  James Schmidt,et al.  The Restructuring of Social and Political Theory , 1978, Telos.

[2]  Z. Stoneman,et al.  Situational Influences on Assessment Performance , 1978, Exceptional children.

[3]  R. Flaugher.  The many definitions of test bias. , 1978 .

[4]  Donald B. Rubin,et al.  Interpersonal expectancy effects: the first 345 studies , 1978, Behavioral and Brain Sciences.

[5]  Cognitive Performance of Kindergarten Children When Tested by Parents and Strangers , 1982 .

[6]  L. Hedges.  Fitting Categorical Models to Effect Sizes from a Series of Experiments , 1982 .

[7]  T. Adorno,et al.  The Positivist dispute in German sociology , 1976 .

[8]  H. Stevenson.  Social Reinforcement of Children's Behavior , 1965 .

[9]  Edward Zigler,et al.  Motivational aspects of changes in IQ test performance of culturally deprived nursery school children. , 1968 .

[10]  R. Nichols.  1: Policy Implications of the IQ Controversy , 1978 .

[11]  S. Feldman,et al.  Factors Mediating the Effects of Enhanced Rapport on Children's Performance. , 1971 .

[12]  Victoria Seitz,et al.  Motivational Factors in the Performance of Economically Disadvantaged Children on the Peabody Picture Vocabulary Test. , 1973 .

[13]  T. Field.  Ecological variables and examiner biases in assessing handicapped preschool children. , 1981, Journal of pediatric psychology.

[14]  Edith Lucille Marine.  The effect of familiarity with the examiner upon Stanford-Binet test performance , 1929 .

[15]  Irwin G. Sarason,et al.  Test anxiety : theory, research, and applications , 1980 .

[16]  A. Thomas,et al.  Examiner effect in IQ testing of Puerto Rican working-class children. , 1971, The American journal of orthopsychiatry.

[17]  E. Sacks.  Intelligence scores as a function of experimentally established social relationships between child and examiner. , 1952, Journal of abnormal psychology.

[18]  Anna M. Dailey,et al.  The Effect of Examiners' Personal Familiarity and Professional Experience on Handicapped Children's Test Performance. , 1985 .

[19]  Edward D. Fiscus.  The effects of pre-test information on school psychologists' scoring of the Wechsler Intelligence Scale for children / , 1975 .

[20]  Victoria Seitz,et al.  Effects of Place of Testing on the Peabody Picture Vocabulary Test Scores of Disadvantaged Head Start and Non-Head Start Children. , 1975 .

[21]  INTELLIGENCE TESTING AS A SOCIAL ACTIVITY11I am grateful for comments made by Courtney Cazden, Aaron Cicourel, Kenneth Leiter, and Hugh Mehan. My wife, Ruth Cohen Roth, has assisted me in more practical and intellectual ways than I could ever acknowledge. , 1974 .

[22]  James A. Kulik,et al.  Effects of Coaching Programs on Achievement Test Performance , 1983 .

[23]  The Influence of Nonintellective Factors on the IQ Scores of Middle- and Lower-Class Children. , 1971 .

[24]  Aaron V. Cicourel,et al.  Language Use and School Performance , 1974 .

[25]  L. Tyler.  The psychology of human differences. , 1947 .

[26]  J. Masling.  The effects of warm and cold interaction on the interpretation of a projective protocol. , 1957, Journal of projective techniques.

[27]  有恒 続,et al.  実験者-被験者間のRapportについての一研究 精神薄弱児の知能測定を通して , 1956 .

[28]  E. Miller,et al.  Headache , 1980 .

[29]  R. L. Carpenter,et al.  Elicitor effects on the language obtained from young language-impaired children. , 1978, The Journal of speech and hearing disorders.

[30]  Variations in Wisc Performances as Influenced by Differences in Pretest Rapport. , 1966, The Journal of general psychology.

[31]  Larry V. Hedges,et al.  Advances in statistical methods for meta-analysis , 1984 .

[32]  A. Greenwald.  Consequences of Prejudice Against the Null Hypothesis , 1975 .

[33]  J. Dusek,et al.  The Bases of Teacher Expectancies: A Meta-Analysis. , 1983 .

[34]  J. Millham,et al.  Effects of Age, Sex, Systematic Conceptual Learning, Acquisition of Learning Sets, and Programmed Social Interaction on the Intellectual and Conceptual Development of Preschool Children from Poverty Backgrounds. , 1971 .

[35]  E. Mishler.  Meaning in Context: Is There Any Other Kind? , 1979 .

[36]  Randolph E. Sarnacki,et al.  An Examination of Test-Wiseness In the Cognitive Test Domain , 1979 .

[37]  William Labov,et al.  THE LOGIC OF NONSTANDARD ENGLISH , 1970 .

[38]  L. Hedges.  Estimation of effect size from a series of independent experiments. , 1982 .

[39]  Participants' Verbal and Nonverbal Behavior in Familiar and Unfamiliar Test Conditions: An Exploratory Analysis , 1983 .

[40]  Test performance of language-handicapped children with familiar and unfamiliar examiners. , 1983, The Journal of psychology.

[41]  C. V. Edlund.  The effect on the test behavior of children, as reflected in the I.Q. scores, when reinforced after each correct response. , 1972, Journal of applied behavior analysis.

[42]  R. Bernstein.  The restructuring of social and political theory , 1978 .

[43]  What the Test Score Really Reflects: Observations of Teacher Behavior During Standardized Achievement Test Administration. , 1981 .

[44]  M. Schoen.  The Moral Judgment of the Child. , 1933 .

[45]  G. Brody,et al.  A Further Examination of Motivational Influences on Disadvantaged Minority Group Children's Intelligence Test Performance. , 1977 .

[46]  J. Hersh.  Effects of referral information on testers. , 1971, Journal of consulting and clinical psychology.

[47]  A. Rosenkrantz,et al.  The Influence of Prior Contact between Child Subjects and Adult Experimenters on Subsequent Child Performance , 1974 .

[48]  T. Ayllon,et al.  Effects of reinforcement on standardized test performance. , 1972, Journal of applied behavior analysis.

[49]  A. Kazdin,et al.  Artifact, bias, and complexity of assessment: the ABCs of reliability. , 1977, Journal of applied behavior analysis.

[50]  R H SCHAPPE,et al.  THE EXPERIMENTER EFFECT. , 1965, Psychological bulletin.

[51]  Anna M. Dailey,et al.  Bias in the Assessment of Handicapped Children , 1985 .

[52]  Karl R. White,et al.  Teacher-Student Interaction Patterns in Classrooms with Mainstreamed Mildly Handicapped Students , 1982 .

[53]  H. Stevenson,et al.  Use of rate as a measure of response in studies of social reinforcement. , 1966, Psychological bulletin.

[54]  H. Schroeder,et al.  Examiner Bias: A Determinant of Children's Verbal Behavior on the WISC. , 1972 .

[55]  D. Irons.  The effect of familiarity with the examiner on WISC‐r verbal, performance, and full scale scores , 1981 .

[56]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[57]  R. Dana,et al.  Self-Help for Male WISC Examiners by Pretest Exposure to Children , 1980 .

[58]  Lynn S. Fuchs,et al.  Examiner Accuracy during Protocol Completion , 1984 .

[59]  T. Christiansen,et al.  A Comparison of Anglo-American and Spanish-American Children on the WISC. , 1970 .

[60]  L. Hedges.  Distribution Theory for Glass's Estimator of Effect size and Related Estimators , 1981 .

[61]  Harris Cooper,et al.  Scientific Guidelines for Conducting Integrative Research Reviews , 1982 .

[62]  Florence Clark,et al.  THE ILLINOIS TEST OF PSYCHOLINGUISTIC ABILITIES , 1982 .

[63]  Michael Cole,et al.  Comparative Studies of How People Think: An Introduction , 1981 .

[64]  R. Rosenthal.  Combining results of independent studies. , 1978 .

[65]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[66]  A. Mitscherlich.  Group Psychology and the Analysis of the Ego , 2020, Nature.

[67]  Effects of Reinforcement and Training on Title I Students' Group Standardized Test Performance. , 1981 .

[68]  H. Mehan.  Structuring School Structure , 1978 .

[69]  Effects of Examiner Familiarity and Task Characteristics on Speech- and Language-Impaired Children's Test Performance , 1984 .

[70]  Elisha Y. Babad,et al.  Bias in Scoring the WISC Subtests. , 1975 .