Design and analysis in task-based language assessment

In task-based language assessment (TBLA) language use is observed in settings that are more realistic and complex than in discrete skills assessments, and which typically require the integration of topical, social and/or pragmatic knowledge along with knowledge of the formal elements of language. But designing an assessment is not accomplished simply by determining the settings in which performance will be observed. TBLA raises questions of just how to design complex tasks, evaluate students’ performances and draw valid conclusions therefrom. This article examines these challenges from the perspective of ‘evidence-centred assessment design’. The main building blocks are student, evidence and task models, with tasks to be administered in accordance with an assembly model. We describe these models, show how they are linked and assembled to frame an assessment argument and illustrate points with examples from task-based language assessment.

[1]  Brian W. Junker,et al.  Applications and Extensions of MCMC in IRT: Multiple Item Types, Missing Data, and Rated Responses , 1999 .

[2]  E. Porteri,et al.  Delayed development of hypertension after short-term nitrendipine treatment. , 1994, Hypertension.

[3]  Peter B. Mosenthal,et al.  Defining the expository discourse continuum , 1985 .

[4]  Eric T. Bradlow,et al.  A Bayesian random effects model for testlets , 1999 .

[5]  S. Embretson.  A cognitive design system approach to generating valid tests : Application to abstract reasoning , 1998 .

[6]  Bert F. Green,et al.  In defense of measurement. , 1978 .

[7]  R. Shavelson.  Performance Assessments: Political Rhetoric and Measurement Reality , 1992 .

[8]  Geoff Brindley,et al.  THE PROMISE AND THE CHALLENGE , 2012 .

[9]  R. Lefkowitz,et al.  Myocardial expression of a constitutively active alpha 1B-adrenergic receptor in transgenic mice induces cardiac hypertrophy. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[10]  R. D. Bock,et al.  Adaptive EAP Estimation of Ability in a Microcomputer Environment , 1982 .

[11]  P. Robinson.  Task complexity, task difficulty, and task production: exploring interactions in a componential framework , 2001 .

[12]  Russell G. Almond,et al.  On the Roles of Task Model Variables in Assessment Design. , 1999 .

[13]  M. Mulvany,et al.  Small artery structure in hypertension. Dual processes of remodeling and growth. , 1993, Hypertension.

[14]  J. Faber,et al.  Regulation of Vascular Smooth Muscle Growth by -Adrenoreceptor Subtypes in Vitro and in Situ(*) , 1995, The Journal of Biological Chemistry.

[15]  P. Skehan.  语言学习认知法 = A cognitive approach to language learning , 1998 .

[16]  M. Mulvany,et al.  Structure and function of small arteries. , 1990, Physiological reviews.

[17]  M. Mulvany,et al.  Histology of subcutaneous small arteries from patients with essential hypertension. , 1993, Hypertension.

[18]  R. Shavelson,et al.  Research news and Comment: Performance Assessments , 1992 .

[19]  M. Piascik,et al.  Expression of multiple alpha1-adrenoceptors on vascular smooth muscle: correlation with the regulation of contraction. , 1999, The Journal of pharmacology and experimental therapeutics.

[20]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[21]  Karen E. Breiner-Sanders,et al.  ACTFL Proficiency Guidelines—Speaking: Revised 1999 , 2000 .

[22]  Lyle F. Bachman.  语言测试要略 = Fundamental considerations in language testing , 1990 .

[23]  Lyle F. Bachman,et al.  The Evaluation of Communicative Language Proficiency: A Critique of the ACTFL Oral Interview , 1986 .

[24]  Irwin S. Kirsch,et al.  Literacy, profiles of America's young adults , 1986 .

[25]  R. Almond,et al.  Making Sense of Data From Complex Assessments , 2002 .

[26]  J. Ross,et al.  Elevated blood pressure and enhanced myocardial contractility in mice with severe IGF-1 deficiency. , 1996, The Journal of clinical investigation.

[27]  Russell G. Almond,et al.  Graphical Models and Computerized Adaptive Testing , 1999 .

[28]  Martijn P. F. Berger,et al.  A Review of Selection Methods for Optimal Test Design. Research Report 94-4. , 1994 .

[29]  J. Frederiksen,et al.  A Systems Approach to Educational Testing , 1989 .

[30]  T. McNamara.  Measuring Second Language Performance , 1996 .

[31]  Russell G. Almond,et al.  A cognitive task analysis with implications for designing simulation-based performance assessment☆ , 1999 .

[32]  Donald B. Rubin,et al.  The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. , 1974 .

[33]  M. Zuscik,et al.  Hypotension, Autonomic Failure, and Cardiac Hypertrophy in Transgenic Mice Overexpressing the α1B-Adrenergic Receptor* , 2001, The Journal of Biological Chemistry.

[34]  J. Ross,et al.  Loss of a gp130 Cardiac Muscle Cell Survival Pathway Is a Critical Event in the Onset of Heart Failure during Biomechanical Stress , 1999, Cell.

[35]  Robert L. Linn,et al.  Performance Assessment: Policy Promises and Technical Measurement Standards , 1994 .

[36]  G. Lembo,et al.  Decreased blood pressure response in mice deficient of the alpha1b-adrenergic receptor. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[37]  M. Mulvany,et al.  Cellular Hypertrophy in Mesenteric Resistance Vessels from Renal Hypertensive Rats , 1988, Hypertension.

[38]  Grant Wiggins,et al.  Assessing student performance , 1993 .

[39]  Dorry M. Kenyon,et al.  COMPARING EXAMINEE ATTITUDES TOWARD COMPUTER- ASSISTED AND OTHER ORAL PROFICIENCY ASSESSMENTS , 2001 .

[40]  James Dean Brown,et al.  Designing Second Language Performance Assessments , 1998 .

[41]  S. Messick.  The Interplay of Evidence and Consequences in the Validation of Performance Assessments , 1994 .

[42]  Karen Draney,et al.  Objective measurement : theory into practice , 1992 .

[43]  R. T. Lee,et al.  Integrin-mediated collagen matrix reorganization by cultured human vascular smooth muscle cells. , 1995, Circulation research.

[44]  M. Canale,et al.  TOEFL FROM A COMMUNICATIVE VIEWPOINT ON LANGUAGE PROFICIENCY: A WORKING PAPER , 1985 .

[45]  J. McMurray,et al.  Effects of chronic norepinephrine administration on cardiac function in rats. , 1995, Journal of cardiovascular pharmacology.

[46]  Elaine Tarone,et al.  English for Academic and Technical Purposes: Studies in Honor of Louis Trimble , 1981 .

[47]  G. H. Fischer,et al.  The linear logistic test model as an instrument in educational research , 1973 .

[48]  R. Almond,et al.  Leverage Points for Improving Educational Assessment , 2000 .

[49]  E. Porteri,et al.  Cellular hypertrophy in subcutaneous small arteries of patients with renovascular hypertension. , 2000, Hypertension.

[50]  Bengt Muthen,et al.  Some uses of structural equation modeling in validity studies: Extending IRT to external variables , 1986 .

[51]  Eva Nick,et al.  The dependability of behavioral measurements: theory of generalizability for scores and profiles , 1973 .

[52]  A. Daugherty,et al.  Chronic Angiotensin II Infusion Promotes Atherogenesis in Low Density Lipoprotein Receptor −/− Mice , 1999, Annals of the New York Academy of Sciences.

[53]  Ernesto L. Schiffrin,et al.  Vascular Remodeling in Hypertension: Roles of Apoptosis, Inflammation, and Fibrosis , 2001, Hypertension.

[54]  Howard Gardner,et al.  To Use Their Minds Well: Investigating New Forms of Student Assessment , 1991 .

[55]  S. Embretson.  Test design : developments in psychology and psychometrics , 1985 .

[56]  Russell G. Almond,et al.  A Sample Assessment Using the Four Process Framework. CSE Technical Report 543. , 2001 .

[57]  Raymond J. Adams,et al.  The Multidimensional Random Coefficients Multinomial Logit Model , 1997 .

[58]  Lyle F. Bachman,et al.  语言测试实践 = Language testing in practice , 1998 .

[59]  Geoff Brindley,et al.  Studies in immigrant English language assessment , 2000 .

[60]  C. Yang,et al.  Mechanism of catecholamine-induced proliferation of vascular smooth muscle cells. , 1998, Circulation.

[61]  A. Mark,et al.  The sympathetic nervous system in hypertension: a potential long-term regulator of arterial pressure. , 1996, Journal of hypertension. Supplement : official journal of the International Society of Hypertension.