Computerized long‐menu questions as an alternative to open‐ended questions in computerized assessment

To optimally avoid cueing effects and computer scoring problems in computerized examinations a computerized long‐menu question (CLM) was developed. This question type was compared to open‐ended questions in one treatment group and to multiple‐choice questions in another treatment group. Also, scores were compared to self‐perceived computer anxiety of the participants. CLMs yield comparable scores to open‐ended questions, but the scores differ significantly from those on multiple‐choice tests. Correlations in the first comparison (CLMs with multiple‐choice) were higher than those in the second camparison (CLMs with open‐ended questions). The amount of positive and negative cueing was considerably higher in the first than in the second comparison. Response times of CLMs were higher than those of multiple‐choice questions and open‐ended questions, differing significantly from both. Computer anxiety did not influence the mean scores in either comparison. Therefore, in computerized testing CLMs seem to offer an acceptable replacement of open‐ended questions.

[1]  P. V. West.  A Critical Study of the Right Minus Wrong Method , 1923 .

[2]  R. Harden,et al.  Multiple choice questions: to guess or not to guess , 1976, Medical education.

[3]  E. Hettiaratchi.  A comparison of student performance in two parallel physiology tests in multiple choice and short answer forms , 1978, Medical education.

[4]  D. F. Votaw,et al.  The effect of do-not-guess directions upon the validity of true-false or multiple choice tests. , 1936 .

[5]  Gérard Sabah.  Knowledge Representation and Natural Language Understanding , 1992, AI Commun..

[6]  David B. Swanson,et al.  Assessment of Clinical Competence: Written and Computer-Based Simulations , 1987 .

[7]  G. Webster,et al.  An Evaluation of a Computer Simulation in the Assessment of Physician Competence , 1986 .

[8]  D. Newble,et al.  A comparison of multiple‐choice tests and free‐response tests in examinations of clinical competence , 1979, Medical education.

[9]  W. McCarthy.  An assessment of the influence of cueing items in objective examinations. , 1966, Journal of Medical Education.

[10]  Robert F. Van Naerssen,et al.  A scale for the measurement of subjective probability , 1962 .

[11]  M. Robeson,et al.  A solution to the cueing effects of multiple choice questions: the Un‐Q format , 1993, Medical education.

[12]  Dale E. Mattson,et al.  The Effects of Guessing On the Standard Error of Measurement and the Reliability of Test Scores , 1965 .

[13]  W. Mccall.  A New Kind of School Examination , 1920 .

[14]  A. Rossier.  Letter to the Editor , 1986, Paraplegia.

[15]  G. M. Ruch,et al.  Corrections for chance and "Guess" vs. "Do not guess" instructions in multiple response tests. , 1926 .

[16]  J. Maatsch,et al.  An evaluation of the construct validity of four alternative theories of clinical competence. , 1986, Research in medical education : proceedings of the ... annual Conference. Conference on Research in Medical Education.

[17]  G. Norman,et al.  Factors underlying performance on written tests of knowledge , 1987, Medical education.

[18]  Henk G. Schmidt,et al.  Effects of mobilizing prior knowledge on information processing: Studies of free recall and allocation of study time , 1993 .

[19]  David B. Swanson,et al.  Extended‐matching items: A practical alternative to free‐response questions , 1993 .

[20]  Delpha Hurlburt.  The Relative Value of Recall and Recognition Techniques for Measuring Precise Knowledge of Word Meaning-Nouns, Verbs, Adjectives , 1954 .