Some Limitations of Criterion-Referenced Measurement.

Ebel, Robert L. Some Limitations of Criterion-Referenced Measurement. American Educational Research Association, Washington, D.C.; Michigan State Univ., East Lansing. 2 Mar 70 9p.; Paper presented at American Educational Research Association Convention, Minneapolis, Minnesota, March 2-6, 1970 DRS Price MP-$0.25 HC-$0.9 *Achievement Rating, Achievement Tests, Changing Attitudes: Criteria, *Critical Incidents Method, *Evaluation Methods, Measurement, *Measurement Techniques, Teaching Techniques, *Test Construction, Testing, Tests Contrary to the impression which exists in some quarters, criterion-referenced measurements are no+ a recent development that modern technology has made possible and that effective education requires. The use of criterion-referenced measurements can not be expected to improve significantly our evaluations of educational achievement. The major limitations of criterion-referenced measurements are: (1) they do not tell us all we need to know about. achievement; (2) they are difficult to obtain on any sound basis; and (3) they are necessary for only a small fraction of important educational achievements. It is true that norm-referenced measurements of educational achievement need to have content meaning as well as relative meaning. We need to understand not just that a student excells or is deficient, but what it is that he does well or poorly. However, these meanings and understandings are seldom wholly absent when norm-referenced measures are used. They can be made more obviously present and useful it we choose to do so. (KJ) U,S, DEPARTMEPTor HEALTH, EDUCATION & W,ELFARE OFFICE. OF EDUCATION THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE PERSON OR ORGANIZATION ORIGINATING IT. POINTS OF VIEW OR OPINIONS STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION POSITION OR POLICY. Some Limitations of Criterion-Referenced Measurement* Robert L. Ebel Michigan State University Every mental test is intended to indicate how much of some particular characteristic an individual can demonstrate. To determine and express "how much" one needs a quantitative scale. Even those tests used primarily for categorical pass-fail decisions almost always involve a quantitative scale on which a critical "passing" score has been defined. Because the human characteristics that mental tests seek to measure are often complex and hard to define, appropriate quantitative scales are not easy to establish. Some of the most difficult problems of mental measurements arise in the process of getting a useful scale. The essential difference between norm-referenced and criterion-referenced measurements is in the quantitative scales used to express how much the individual can do. In norm-referenced measurement the scale is usually anchored in the middle on some average level of performance for a particular group of individuals. The units on the scale are usually a function of the distribution of performances above and below the average level. In criterion-referenced measurement the scale is usually anchored at ti.e extremities, a score at the top of the scale indicating complete or perfect mastery of some defined abilities, one at the bottom indicating complete absence of those abilities. The scale units consist of subdivisions of this total scale range. *Prepared for A9RA Symposium, "Criterion-Referenced Measurement: Emerging Issues" Minneapolis, Minnesota, March 1970.