Achievement Differences on Multiple-Choice and Essay Tests in Economics

Multiple-choice and essay tests are the typical test formats used to measure student understanding of economics in college courses. Each type has its features. A multiple-choice (or fixed-response) format allows for a wider sampling of the content because more questions can be given in a testing period. Multiple-choice tests also offer greater efficiency and reliability in scoring than an essay. The major disadvantage of a multiple-choice item is that the fixed responses tend to emphasize recall and encourage guessing. In an essay (or constructed-response) test, students generate responses that have the potential to show originality and a greater depth of understanding of the topic. The essay also provides a written record for assessing the thought processes of the student.1 Despite the claimed differences for each format, little empirical work exists to support the suppositions. If a multiple-choice and an essay test that cover the same material measure the same economic understanding, then the multiple-choice test would be the preferred method for assessment because it is less costly to score and is a more reliable measure of achievement in a limited testing period.2 If, however, an essay test measures unique aspects of economic understanding, then the extra examinee time and substantial scoring costs may be justified. The research evidence from other subjects suggests that there is little difference in the knowledge, skills, or abilities measured by multiple-choice and essay (or constructed-response) tests. A study of Advanced Placement (AP) tests in seven college subjects (calculus, computer science, chemistry, biology, history, French, and music) concluded that "whatever is being measured by the constructed-response section is measured better by the multiple-choice section.... We have never found any test that is composed of an objectively and a subjectively scored section for which this is not true" (Howard Wainer and David Thissen, 1993 p. 116). Similarly, an investigation of the AP exam in computer science found "little support for the stereotype of multiple-choice and free-response formats as measuring substantially different constructs (i.e., trivial factual recognition vs. higher order processes)" (Randy Bennett et al., 1991 p. 89). A review of studies in four domains (writing, word knowledge, reading, and quantitative) was more equivocal about the value of constructed response but concluded that "if differences do exist for any domain, they are very likely to be small" (Ross Traub, 1993 p. 38). Finally, a study tDiscussants: William Greene, New York University; Jane Lillydahl, University of Colorado.