Scores on a yes-no vocabulary test: correction for guessing and response style

The use of yes-no tests seems to be a promising method for measuring the size of receptive vocabulary knowledge of learners of a foreign language. Items in a yes-no test each consist of either a word or a pseudoword. Participants are asked to indicate whether or not they know the meaning of these words. This article attempts to tackle the problem of determining a meaningful score for this type of test. Such a score should contain correction for guessing as well as for participants’ response style. Three possible methods are discussed, but none of these measures appear to apply this type of correction. Signal Detection Theory is applied and a new, more accurate index is suggested. Based on theoretical as well as empirical considerations, recommendations are made about the choice for the index to be used in a yes-no vocabulary test.

[1]  J. Grier,et al.  Nonparametric indexes for sensitivity and bias: computing formulas. , 1971, Psychological bulletin.

[2]  C. Chapelle.  Construct definition and validity inquiry in SLA research , 1999 .

[3]  William Hodos,et al.  Nonparametric index of response bias for use in detection and recognition experiments. , 1970 .

[4]  Donald A. Norman,et al.  A non-parametric analysis of recognition experiments , 1964 .

[5]  Richard C. Anderson,et al.  Reading comprehension and the assessment and acquisition of word knowledge , 1982 .

[6]  D. M. Green,et al.  On the equivalence of two recognition measures of short-term memory. , 1966, Psychological bulletin.

[7]  Y. Hoshino.  A bias in favor of the positive response to high-frequency words in recognition memory , 1991, Memory & cognition.

[8]  P. Meara,et al.  An alternative to multiple choice vocabulary tests , 1987 .

[9]  D S Emmerich,et al.  Receiver-operating characteristics determined under several interaural conditions of listening. , 1968, The Journal of the Acoustical Society of America.

[10]  D. Behrend,et al.  Overextensions in early language comprehension: evidence from a signal detection approach , 1988, Journal of Child Language.

[11]  R. E. Pastore,et al.  SIGNAL DETECTION THEORY: CONSIDERATIONS FOR GENERAL APPLICATION , 1974 .

[12]  Irwin Pollack,et al.  Sampling variability of the area under the ROC-curve and of d'e. , 1969 .

[13]  Colin M. Macleod,et al.  Word frequency effects on recall, recognition, and word fragment completion tests. , 1996, Journal of experimental psychology. Learning, memory, and cognition.