This study analyzed examinee responses to two essay prompts being considered for the Graduate Record Examinations (GRE®) Writing Assessment: “Present your Perspective on an Issue” and “Analyze an Argument.” Forty prompts (20 issue and 20 argument) were administered to over 2,300 students at 26 participating U.S. colleges and universities. Each student wrote two essays in response to either two issue prompts, two argument prompts, or one of each. Results show that the issue and argument writing tasks appear to assess relatively similar constructs, supporting the decision to include both types of prompts in the operational GRE Writing Assessment and to report a single, combined score. The results also support the random administration of prompts without equating adjustments, because within each task type, most of the prompts were comparable in difficulty and no important subgroup interactions with prompt classifications were detected. Finally, from a fairness perspective, the results show there were advantages to administering the issue prompt first and the argument prompt second. The GRE Program used the psychometric information provided by this study to make final design, delivery, and scoring decisions before introducing the operational assessment in the fall of 1999.
[1]
J. R. Landis,et al.
The measurement of observer agreement for categorical data.
,
1977,
Biometrics.
[2]
Brent Bridgeman,et al.
Writing Assessment in Admission to Higher Education: Review and Framework. College Board Report No. 99-3. GRE Board Research Report No. 96-12R.
,
1999
.
[3]
Jacob Cohen.
A Coefficient of Agreement for Nominal Scales
,
1960
.
[4]
D. B. Duncan.
MULTIPLE RANGE AND MULTIPLE F TESTS
,
1955
.
[5]
Donald E. Powers,et al.
Effects of Applying Different Time Limits to a Proposed GRE Writing Test
,
1996
.
[6]
John W. Tukey,et al.
Exploratory Data Analysis.
,
1979
.
[7]
VALIDATING A WRITING TEST FOR GRADUATE ADMISSIONS
,
1996
.