The influence of interlocutor proficiency in a paired oral assessment

The use of pair work in speaking assessment may encourage collaboration in the classroom and has other advantages (Saville & Hargreaves, 1999; Taylor, 2000) but from a measurement perspective, the paired oral format may be problematic because a partner may unfairly influence an examinee's performance or otherwise bias scores. In this study, the influence of interlocutor proficiency on speaking performance was examined in a group of 20 first-year students at a Chinese university. The students were divided into groups of relatively high and low English proficiency and tested once with a partner of similar proficiency and once with a partner of higher or lower proficiency. Interlocutor proficiency level had no observable effect on Rasch analysis ability measures, but lower-level examinees produced more language (words) when working with a higher-level partner. The majority of dyads produced collaborative interactions (Galaczi, 2008), unless an examinee was paired with a much lower-level partner, in which case the interaction tended to be asymmetric. Overall, these data suggest that proficiency differences among examinees need not preclude use of the paired oral test format.

[1]  Michael Milanovic,et al.  Continuity and innovation : revising the Cambridge Proficiency in English Examination, 1913-2002 , 2003 .

[2]  P. Skehan.  语言学习认知法 = A cognitive approach to language learning , 1998 .

[3]  Leo Van Lier,et al.  Reeling, Writhing, Drawling, Stretching, and Fainting in Coils: Oral Proficiency Interviews as Conversation , 1989 .

[4]  Vivien Berry.  Personality Differences and Oral Test Performance , 2007 .

[5]  Liying Cheng,et al.  Description and Examination of the National Matriculation English Test , 2006 .

[6]  James Dean Brown,et al.  Designing Second Language Performance Assessments , 1998 .

[7]  Michael C. Foot.  Relaxing in pairs , 1999 .

[8]  Anne Lazaraton,et al.  A qualitative approach to the validation of oral language tests , 2002 .

[9]  Tim McNamara,et al.  ‘Interaction’ in second language performance assessment: Whose performance?1 , 1997 .

[10]  Merrill Swain,et al.  Researching Pedagogic Tasks: Second Language Learning, Teaching, and Testing , 2013 .

[11]  George Yule,et al.  Resolving Referential Conflicts in L2 Interaction: The Effect of Proficiency and Interactive Role † , 1990 .

[12]  James A. Jones,et al.  Uses of Rasch Modeling in Counseling Psychology Research. , 1998 .

[13]  Evelina D. Galaczi,et al.  Peer–Peer Interaction in a Speaking Test: The Case of the First Certificate in English Examination , 2008 .

[14]  Jack Whalen,et al.  Sociology as a Natural Observational Science@@@Structures of Social Action: Studies in Conversation Analysis. , 1987 .

[15]  Tim McNamara,et al.  Modelling Performance: Opening Pandora's Box , 1995 .

[16]  Meaning Negociation in the Hungarian Oral Proficiency Examination of English , 1998 .

[17]  Elana Shohamy,et al.  The validity of direct versus semi- direct oral tests , 1994 .

[18]  Barry O’Sullivan,et al.  Learner acquaintanceship and oral proficiency test pair-task performance , 2002 .

[19]  Alan Tonkyn,et al.  Measuring spoken language: a unit for all reasons , 2000 .

[20]  Gary J. Ockey,et al.  A many-facet Rasch analysis of the second language group oral discussion task , 2003 .

[21]  K. O’Loughlin,et al.  The Equivalence of Direct and Semi-Direct Speaking Tests , 2001 .

[22]  Sylvia Green,et al.  University of Cambridge Local Examination Syndicate , 1943, The Mathematical Gazette.

[23]  Elaine Tarone,et al.  Interfaces between Second Language Acquisition and Language Testing Research: Research on interlanguage variation: Implications for language testing , 1999 .

[24]  W. Damon,et al.  Critical distinctions among three approaches to peer education , 1989 .

[25]  Annie Brown,et al.  Interviewer variation and the co-construction of speaking proficiency , 2003 .

[26]  Nick Saville,et al.  Assessing speaking in the revised FCE , 1999 .

[27]  Anne Lazaraton,et al.  Interlocutor support in oral proficiency interviews: the case of CASE , 1996 .

[28]  L. Ortega.  Syntactic Complexity Measures and Their Relationship to L2 Proficiency: A Research Synthesis of College-Level L2 Writing. , 2003 .

[29]  Neomy Storch,et al.  Patterns of Interaction in ESL Pair Work , 2002 .

[30]  T. McNamara.  Measuring Second Language Performance , 1996 .

[31]  Michael H. Long,et al.  Three Approaches to Task‐Based Syllabus Design , 1992 .

[32]  Graham Crookes,et al.  The Utterance, and Other Basic Units for Second Language Discourse Analysis. , 1990 .

[33]  Tim McNamara,et al.  "The devil is in the detail": Researching gender issues in language assessment , 2004 .

[34]  E. Galaczi.  PEER-PEER INTERACTION IN A PAIRED SPEAKING TEST: THE CASE OF FCE , 2010 .

[35]  Michael Milanovic,et al.  Discourse Variation in Oral Proficiency Interviews , 1992, Studies in Second Language Acquisition.

[36]  M. R. Espejo.  Applying the Rasch Model: Fundamental Measurement in the Human Sciences , 2004 .

[37]  P. Glover,et al.  Readers respond. Oral testing in pairs - secondary school perspective , 2001 .

[38]  Glenn Fulcher,et al.  Testing Second Language Speaking , 2003 .

[39]  D. Tannen.  New York Jewish Conversational Style , 1981 .