Expertise in evaluating second language compositions

This study (1) assesses whether raters implicitly distinguish students' writing expertise and second language proficiency while evaluating ESL compositions holistically and (2) seeks to describe the decision-making behaviours used by experienced and inexperienced raters in this process. The performance of 7 novice and 6 expert ESL teachers was assessed while they evaluated 12 compositions written by adult students with differing levels of ESL proficiency (intermediate and advanced) and writing expertise (average and professionally experienced writers) in their mother tongues. Multivariate analyses of rating scores indicated that both groups' evaluations distinguished students' second language proficiency and writing skills as separate, non-interacting factors. Descriptive analyses of the raters' concurrent verbal reports revealed 28 common decision-making behaviours, many of which varied significantly in use between novice and expert groups. Implications discuss biases in holistic evaluations of second language compositions, aspects of expertise in this skill, and potential uses of this research for the training of composition raters and student-teachers.

[1]  Problems in Stabilizing the Judgment Process. , 1980 .

[2]  Michelene T. H. Chi,et al.  Expertise in Problem Solving. , 1981 .

[3]  Kyle Perkins,et al.  On the Use of Composition Scoring Techniques, Objective Measures, and Objective Tests to Evaluate ESL Writing Ability , 1983 .

[4]  T. Homburg.  Holistic Evaluation of ESL Compositions: Can It Be Validated Objectively? , 1984 .

[5]  Alan C. Purves.  In Search of an Internationally-Valid Scheme for Scoring Compositions , 1984 .

[6]  Alistair Cumming.  Responding to the Writing of ESL Students. , 1985 .

[7]  W. Kintsch.  Learning From Text , 1986, Knowing, Learning, and Instruction.

[8]  J. Greeno,et al.  The cognitive skill of teaching. , 1986 .

[9]  Robert J. Yinger,et al.  Learning the Language of Practice , 1987 .

[10]  Sarah Warshauer Freedman,et al.  Response to Student Writing , 1987 .

[11]  Alister Cumming,et al.  Professor's Ratings of Language Use and Rhetorical Organizations in ESL Compositions , 1987 .

[12]  M. Scardamalia,et al.  The psychology of written composition , 1987 .

[13]  W. Kintsch.  The role of knowledge in discourse comprehension: a construction-integration model. , 1988, Psychological review.

[14]  P. Alexander,et al.  The Interaction of Domain-Specific and Strategic Knowledge in Academic Performance , 1988 .

[15]  Dona M. Kagan,et al.  Teaching as Clinical Problem Solving: A Critical Examination of the Analogy and Its Implications , 1988 .

[16]  T. Santos.  Professors' Reactions to the Academic Writing of Nonnative-Speaking Students. , 1988 .

[17]  Alister Cumming,et al.  Student teachers' conceptions of curriculum: Toward an understanding of language-teacher development , 1989 .

[18]  Alister Cumming,et al.  Reading and summarizing challenging texts in first and second languages , 1989 .

[19]  D. Perkins,et al.  Are Cognitive Skills Context-Bound? , 1989 .

[20]  Alister Cumming,et al.  WRITING EXPERTISE AND SECOND LANGUAGE PROFICIENCY , 1989 .

[21]  Reader Comprehension and Holistic Assessment of Second Language Writing Proficiency , 1989 .

[22]  D. Freeman.  Teacher Training, Development, and Decision Making: A Model of Teaching and Related Strategies for Language Teacher Education. , 1989 .

[23]  Charles A. Perfetti,et al.  There are Generalized Abilities and One of Them is Reading , 2018, Knowing, Learning, and Instruction.