The influence of prompt on group oral tests

This paper reports an investigation into how the prompt may influence the discourse of group oral tests. The group oral test, in which three or four participants are rated on their ability to discuss a prompt, is a format for assessing the spoken ability of language learners. In this study, 141 Japanese university students were videoed in 41 group orals of three or four test-takers. Although the four different prompts written for the test were supposed to be of equal difficulty, they were found to be substantially different in the type and number of questions that comprised them. Analysis of the transcribed interactions revealed significant differences in turns taken, syntactic complexity and fluency of the interactions they elicited. A qualitative examination revealed that the two prompts that elicited longer, more complex turns did so by encouraging test-takers to explain their family circumstances or speculate about their future. Prompts with more factual content elicited shorter, less complex turns and the prompt that test-takers responded to with the least fluency required the test-takers to talk about a more personal subject. The implications for rating and creating prompts are discussed, and the need to tailor them to the purpose of the test.

[1]  Walter Loban,et al.  Language Development: Kindergarten through Grade Twelve. NCTE Committee on Research Report No. 18. , 1976 .

[2]  Walter Schneider,et al.  Controlled and Automatic Human Information Processing: 1. Detection, Search, and Attention. , 1977 .

[3]  D. Hargreaves,et al.  Teachers’ questions: open, closed and half‐open , 1984 .

[4]  P. Lennon.  Investigating Fluency in EFL: A Quantitative Approach* , 1990 .

[5]  H. Riggenbach.  Toward an understanding of fluency: A microanalysis of nonnative speaker conversations , 1991 .

[6]  R. Hawkins,et al.  The development of fluency in advanced learners of French , 1996 .

[7]  S. Eggins,et al.  Analysing Casual Conversation , 1996 .

[8]  W. Labov,et al.  Narrative analysis: Oral versions of personal experience. , 1997 .

[9]  Lyle F. Bachman,et al.  语言测试实践 = Language testing in practice , 1998 .

[10]  Alan Tonkyn,et al.  Measuring spoken language: a unit for all reasons , 2000 .

[11]  Merrill Swain,et al.  Examining dialogue: another approach to content specification and to validating inferences drawn from test scores , 2001 .

[12]  P. Robinson.  Task complexity, task difficulty, and task production: exploring interactions in a componential framework , 2001 .

[13]  Glenn Fulcher,et al.  Task difficulty in speaking tests , 2003 .

[14]  Gary J. Ockey,et al.  A many-facet Rasch analysis of the second language group oral discussion task , 2003 .

[15]  Andy Field,et al.  Discovering statistics using SPSS, 2nd ed. , 2005 .

[16]  J. Norton,et al.  The paired format in the Cambridge Speaking Tests , 2005 .

[17]  Barbara A. Wasik,et al.  The Effects of a Language and Literacy Intervention on Head Start Children and Teachers , 2006 .

[18]  Lianzhen He,et al.  A corpus-based investigation into the validity of the CET-SET group discussion , 2006 .

[19]  A. V. Moere.  Validity Evidence in a University Group Oral Test. , 2006 .

[20]  Folkert Kuiken,et al.  The influence of complexity in monologic versus dialogic tasks in Dutch L2 , 2007 .

[21]  Peter Robinson,et al.  Task complexity, theory of mind, and intentional reasoning: Effects on L2 speech production, interaction, uptake and perceptions of task difficulty , 2007 .

[22]  Naoko Taguchi.  Task Difficulty in Oral Speech Act Production , 2007 .

[23]  Extroversion and group oral performance: A mixed quantitative and discourse analysis approach , 2008 .

[24]  Evelina D. Galaczi,et al.  Peer–Peer Interaction in a Speaking Test: The Case of the First Certificate in English Examination , 2008 .

[25]  Pauline Foster,et al.  Task Design and Second Language Performance: The Effect of Narrative Type on Learner Output , 2008 .

[26]  Marysia Johnson.  The Art of Non-conversation , 2008 .

[27]  Pauline Foster,et al.  Native Speakers and Task Performance: Comparing Effects on Complexity, Fluency, and Lexical Diversity. , 2009 .

[28]  G. Ockey.  The effects of group members' personalities on a test taker's L2 group oral discussion test scores , 2009 .

[29]  R. Ellis.  The Differential Effects of Three Types of Task Planning on the Fluency, Complexity, and Accuracy in L2 Oral Production , 2009 .

[30]  Jenifer Larson-Hall,et al.  A Guide to Doing Statistics in Second Language Research Using SPSS , 2009 .

[31]  Peter Skehan,et al.  Modelling Second Language Performance: Integrating Complexity, Accuracy, Fluency, and Lexis , 2009 .

[32]  Lindsay Brooks,et al.  Interacting in pairs in a test of oral proficiency: Co-constructing a better performance , 2009 .

[33]  Chi Cheung Ruby 楊紫嬙 Yang,et al.  Teacher questions in second language classrooms: An investigation of three case studies , 2010 .

[34]  Zhengdong Gan,et al.  Interaction in group oral assessment: A case study of higher- and lower-scoring students , 2010 .

[35]  Fumiyo Nakatsuhara.  Effects of test-taker characteristics and the number of participants in group oral tests , 2011 .

[36]  Pauline Foster,et al.  Task Design and Second Language Performance: The Effect of Narrative Type on Learner Output: Task Design and L2 Performance , 2011 .

[37]  P. Robinson.  Task‐Based Language Learning: A Review of Issues , 2011 .

[38]  Mable B. Kinzie,et al.  Teacher question and student response with regard to cognition and language use , 2012 .