A Conversation Analysis—Informed Test of L2 Aural Pragmatic Comprehension

Speech act theory–based, second language pragmatics testing (SLPT) raises test-validation issues owing to a lack of correspondence with empirical conversational data. On the assumption that conversation analysis (CA) provides a more accurate account of language use, it is suggested that CA serve as a more empirically valid basis for SLPT development. The current study explores this notion by administering a pilot CA-informed test (CAIT) of listening comprehension to learners of English as second language (ESL) and to a control group of native speakers of English. The listening CAIT protocol involved participants' addressing multiple-choice items after listening to audiotaped conversational sequences derived from the CA literature. Statistical analyses of pilot-test responses, correlations of test score with participant demographic variables, and CA-informed, qualitative analyses of nonnative and native speaker responses with reference to operationalized pragmatic norms provided tentative evidence that the CAIT aural-comprehension measure possesses some utility in SLPT.

[1]  G. Kasper,et al.  Cross-Cultural Pragmatics: Requests and Apologies , 1991 .

[2]  Pamela A. Moss,et al.  Can There Be Validity Without Reliability? , 1994 .

[3]  Carol A. Chapelle,et al.  VALIDITY IN LANGUAGE ASSESSMENT , 1999, Annual Review of Applied Linguistics.

[4]  James Dean Brown,et al.  Developing prototypic measures of cross-cultural pragmatics , 1998 .

[5]  D. Maynard,et al.  Standardized Testing as an Interactional Phenomenon. , 1990 .

[6]  Z. Dörnyei,et al.  Do Language Learners Recognize Pragmatic Violations? Pragmatic Versus Grammatical Awareness in Instructed L2 Learning. , 1998 .

[7]  P. Horst.  Correcting the Kuder-Richardson reliability for dispersion of item difficulties. , 1953, Psychological bulletin.

[8]  Anita M. Pomerantz.  Agreeing and disagreeing with assessments: some features of preferred/dispreferred turn shapes , 1984 .

[9]  Alex Lascarides,et al.  Indirect Speech Acts , 2001, Synthese.

[10]  Fred Davidson.  The language tester's statistical toolbox , 2000 .

[11]  S. Blum-Kulka,et al.  Too Many Words: Length of Utterance and Pragmatic Failure , 1986, Studies in Second Language Acquisition.

[12]  E. Schegloff,et al.  A simplest systematics for the organization of turn-taking for conversation , 1974 .

[13]  Jacob L. Mey,et al.  Pragmatics: An Introduction , 2001 .

[14]  George A. Miller,et al.  Language and Communication , 1951 .

[15]  Gary A. Cziko.  Psychometric and Edumetric Approaches to Language Testing: Implications and Applications , 1981 .

[16]  J. Austin.  How to do things with words , 1962 .

[17]  Leonard S. Cahen,et al.  Educational Testing Service , 1970 .

[18]  R. Schmidt.  The role of consciousness in second language learning , 1990 .

[19]  James Dean Brown.  Pragmatics in Language Teaching: Pragmatics tests: Different purposes, different tests , 2001 .

[20]  Kyu-hyun Kim,et al.  Confirmation Sequences as Intercational Resources in Korean Language Proficiency Interviews , 1998 .

[21]  Lawrence F. Bouton.  A Cross-cultural Study of Ability to Interpret Implicatures in English. , 1988 .

[22]  M. Swain,et al.  THEORETICAL BASES OF COMMUNICATIVE APPROACHES TO SECOND LANGUAGE TEACHING AND TESTING , 1980 .

[23]  An investigation of a criterion- referenced test using G-theory, and factor and cluster analyses , 1992 .

[24]  Andrea Golato.  Studying Compliment Responses: A Comparison of DCTs and Recordings of Naturally Occurring Talk , 2003 .

[25]  E. Schegloff.  Reflections on Quantification in the Study of Conversation , 1993 .

[26]  Dale A. Koike.  Transfer of pragmatic competence and suggestions in Spanish foreign language learning , 2006 .

[27]  Lawrence F. Bouton.  The Interpretation of Implicature in English by NNS: Does It Come Automatically--Without Being Explicitly Taught?. , 1992 .

[28]  T. Hudson.  Relationships among IRT item discrimination and item fit indices in criterion-referenced language testing , 1991 .

[29]  Anne Lazaraton,et al.  Preference Organization in Oral Proficiency Interviews: The Case of Language Ability Assessments , 1997 .

[30]  Kenneth R. Rose.  Speech acts and questionnaires: The effect of hearer response , 1992 .

[31]  Alexander Z. Guiora,et al.  EMPATHY AND SECOND LANGUAGE LEARNING1 , 1972 .

[32]  Andrea Tyler,et al.  Re-analyzing the OPI: How Much Does It Look Like Natural Conversation? , 1998 .

[33]  Lyle F. Bachman.  语言测试要略 = Fundamental considerations in language testing , 1990 .

[34]  RAYMOND W. GIBBS,et al.  Literal Meaning and Psychological Theory , 1984, Cogn. Sci..

[35]  Kenneth R. Rose,et al.  Pragmatics in Language Teaching: Name index , 2001 .

[36]  Emanuel A. Schegloff,et al.  Presequences and indirection: Applying speech act theory to ordinary conversation , 1988 .

[37]  G. Kasper,et al.  Pragmatic Development in a Second Language , 2003 .

[38]  Kathleen Bardovi-Harlig,et al.  Input in an Institutional Setting. , 1996 .

[39]  Steven J. Ross,et al.  A comparative task-in-interaction analysis of OPI backsliding , 2007 .

[40]  Anita M. Pomerantz.  Compliment Responses: Notes on the Co-operation of Multiple Constraints , 1978, Asking and Telling in Conversation.

[41]  Lawrence F. Bouton.  Pragmatics and Language Learning. , 1996 .

[42]  S. Blum-Kulka.  Learning to Say What You Mean in a Second Language: A Study of the Speech Act Performance of Learners of Hebrew as a Second Language1 , 1982 .