Investigating Writing Sub-skills in Testing English as a Foreign Language: A Structural Equation Modeling Study

This study investigates the validity of a writing model proposed as the underlying structure of the writing skill in English as a foreign language (EFL). Four writing prompts were administered to 178 Iranian EFL learners. The scripts were then scored according to writing benchmarks similar to the IELTS Writing criteria but narrower in scope. After inter- and intra-rater reliability analysis, a three-factor model was posited for validation. Structural modeling of the sub-skills revealed the two sub-skills of Idea Arrangement and Communicative Quality are psychometrically inseparable, but the Vocabulary and Grammar sub-skills proved to have good measurement properties. Using parcel indicators, a two-factor model was then evaluated which had the best fit and parsimony. The researcher concludes Idea Arrangement and Communicative Quality appear to have similar conceptual and theoretical foundations and should be considered the elements of one measuring criterion. Further research is required to support this finding. [1]

[1]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[2]  J. D. Brown,et al.  A CATEGORICAL INSTRUMENT FOR SCORING SECOND LANGUAGE WRITING SKILLS , 1984 .

[3]  Peter Mickan,et al.  Study of response validity of the IELTS writing subtest , 2000 .

[4]  Gillian Wigglesworth,et al.  Task.design in IELTS academic writing task 1: The effect of quantity and manner of presentation of information on candidate writing , 2003 .

[5]  S. Messick.  Meaning and Values in Test Validation: The Science and Ethics of Assessment , 1989 .

[6]  P. Barrett.  Structural equation modelling : Adjudging model fit , 2007 .

[7]  Gusti Gede Astika,et al.  Analytical Assessments of Foreign Students' Writing , 1993 .

[8]  Liz Hamp-Lyons.  Assessing Second Language Writing in Academic Contexts , 1991 .

[9]  E. Schaefer.  Rater bias patterns in an EFL writing assessment , 2008 .

[10]  Janne Morton,et al.  Authenticity in the IELTS academic writing & module test: a comparative study of Task 2 items & University research , 1999 .

[11]  H. Jacobs.  Testing Esl Composition: A Practical Approach , 1981 .

[12]  Leslie Lamport,et al.  Basic Concepts , 1981, Advanced Course: Distributed Systems.

[13]  G. Heath.  Writing , 1971, Veterinary Record.

[14]  Tim Moore,et al.  Authenticity in the IELTS Academic Module Writing Test: a comparative study of Task 2 items and university assignments , 1999 .

[15]  Dana R. Ferris,et al.  Treatment of Error in Second Language Student Writing , 2002 .

[16]  T. Little,et al.  To Parcel or Not to Parcel: Exploring the Question, Weighing the Merits , 2002 .

[17]  Johanne Myles,et al.  Second Language Writing and Research: The Writing Process and Error Analysis in Student Texts. , 2002 .

[18]  Lyle F. Bachman.  语言测试要略 = Fundamental considerations in language testing , 1990 .

[19]  R. P. McDonald,et al.  Principles and practice in reporting structural equation analyses. , 2002, Psychological methods.

[20]  Raphael Gamaroff,et al.  Rater reliability in language assessment: the bug of all bears , 2000 .

[21]  Khaled Barkaoui,et al.  Rating scale impact on EFL essay marking: A mixed-method study , 2007 .

[22]  Alasdair Archibald,et al.  TARGETING L2 WRITING PROFICIENCIES: INSTRUCTION AND AREAS OF CHANGE IN STUDENTS' WRITING OVER TIME , 2001 .

[23]  Averil Coxhead,et al.  Preparing writing teachers to teach the vocabulary and grammar of academic prose , 2007 .

[24]  James P. Lantolf,et al.  Sociocultural theory and the teaching of second languages , 2008 .

[25]  Vanessa Jakeman,et al.  Cambridge practice tests for IELTS 1 , 1996 .

[26]  Janne Morton,et al.  Dimensions of difference: a comparison of university writing and IELTS writing , 2005 .

[27]  Jayakaran Mukundan,et al.  Analytic assessment of writing: diagnosing areas of strength and weakness in the writing of TESL undergraduate students , 2009 .

[28]  S. Messick.  MEANING AND VALUES IN TEST VALIDATION: THE SCIENCE AND ETHICS OF ASSESSMENT1 , 1988 .

[29]  T. McNamara.  Measuring Second Language Performance , 1996 .

[30]  Cyril J. Weir,et al.  Communicative Language Testing , 1991 .

[31]  Donald A. Daiker.  Sentence-Combining and Syntactic Maturity in Freshman English. , 1978 .

[32]  T. Homburg.  Holistic Evaluation of ESL Compositions: Can It Be Validated Objectively? , 1984 .

[33]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[34]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[35]  Anne Margaret Smith,et al.  Documenting features of written language production typical at different IELTS band score levels , 2007 .

[36]  T. McNamara.  Item Response Theory and the validation of an ESP test for health professionals , 1990 .

[37]  Ute Knoch,et al.  ‘Little coherence, considerable strain for reader’: A comparison between two rating scales for the assessment of coherence , 2007 .

[38]  Peter Mickan,et al.  Text analysis and the assessment of academic writing , 2003 .

[39]  Dianne M. Finkelstein,et al.  A Beginner's Guide to Structural Equation Modeling , 2005, Technometrics.

[40]  H. Chow.  "Holistic assessment : what goes on in the raters' minds?" , 1997 .

[41]  Peter Mickan.  'What's your score?': An investigation into language descriptors for rating written performance , 2003 .

[42]  Jeremy Harmer,et al.  How to Teach Writing , 2004 .

[43]  Sara Cushing Weigle,et al.  Effects of training on raters of ESL compositions , 1994 .

[44]  Lin Lougheed.  Barron's How to Prepare for the Computer-Based Toefl Essay: Test of English As a Foreign Language , 2000 .