The Edumetric Quality of New Modes of Assessment: Some Issues and Prospects

Assessment has played a crucial role in education and training since formal education commenced. Certainly, assessment of learning has been seen as the cornerstone of the learning process since it reveals whether the learning process results in success or not. For many decades, teachers, trainers and assessment institutes were the only partners seen as crucial in the assessment event. Students were seen as subjects who were to be tested without having any influence on any other aspect of the assessment process. Recently, several authors have called our attention to what is often termed ‘new modes of assessment’ and ‘assessment for learning’. They stress that assessment can be used as ameans to reinforce learning, to drive learning and to support learning – preferably when assessment is not perceived by students as a threat, an event they have to fear, the sword of Damocles. These authors also emphasize that the way we assess students should be congruent with the way we teach and the way students learn within a specific learning environment. As such, the ‘new assessment culture’ makes a plea for integrating instruction and assessment. Some go even further: students can play a role in the construction of assessment tasks, the development of assessment criteria, and the scoring of performance can be shared amongst students and teachers. New modes of assessment that arise from such thinking are, for example, 908 or 1808 feedback, writing samples, exhibitions, portfolio assessments, peerand co-assessment, project and product assessments, observations, textand curriculum-embedded questions, interviews, and performance assessments. It is widely accepted that these new modes of assessment lead to a number of benefits in terms of the learning process: encouraging thinking, increasing learning and increasing students’ confidence (Falchikov, 1986, 1995).

[1]  Leonard S. Cahen,et al.  Educational Testing Service , 1970 .

[2]  Donald B. Rubin,et al.  The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. , 1974 .

[3]  F. Marton,et al.  ON QUALITATIVE DIFFERENCES IN LEARNING: I—OUTCOME AND PROCESS* , 1976 .

[4]  Lewis Elton,et al.  Trends in research on student learning , 1979 .

[5]  John D. Bain,et al.  Contextual dependence of learning approaches: The effects of assessments. , 1984 .

[6]  N. Frederiksen The real test bias: Influences of testing on teaching and learning. , 1984 .

[7]  Harold Gulliksen,et al.  Creating Better Classroom Tests. , 1985 .

[8]  Nancy Falchikov,et al.  PRODUCT COMPARISONS AND PROCESS BENEFITS OF COLLABORATIVE PEER GROUP AND SELF ASSESSMENTS , 1986 .

[9]  T. Crooks The Impact of Classroom Evaluation Practices on Students , 1988 .

[10]  Darrell L. Butler A critical evaluation of software for experiment development in research and teaching , 1988 .

[11]  C. Desforges,et al.  Testing and Assessment , 1989 .

[12]  Lee J. Cronbach,et al.  Construct validation after thirty years. , 1989 .

[13]  S. Messick Meaning and Values in Test Validation: The Science and Ethics of Assessment , 1989 .

[14]  J. Frederiksen,et al.  A Systems Approach to Educational Testing , 1989 .

[15]  D. Boud Assessment and the promotion of academic values , 1990 .

[16]  Edward H. Haertel New Forms of Teacher Assessment , 1991 .

[17]  Stephen B. Dunbar,et al.  Complex, Performance-Based Assessment: Expectations and Validation Criteria , 1991 .

[18]  K. Trigwell,et al.  RELATING APPROACHES TO STUDY AND QUALITY OF LEARNING OUTCOMES AT THE COURSE LEVEL , 1991 .

[19]  Michael W. Kirst,et al.  Interview on Assessment Issues with Lorrie Shepard. , 1991 .

[20]  Edward H. Haertel Chapter 1: New Forms of Teacher Assessment , 1991 .

[21]  Michael Prosser,et al.  Improving the quality of student learning: the influence of learning context and student approaches to learning on learning outcomes , 1991 .

[22]  Leona Schauble Book Reviews: Playing With Matches , 1991 .

[23]  Paul J. Feltovich,et al.  Learning, Teaching and Testing for Complex Conceptual Understanding. , 1991 .

[24]  C. Tan An evaluation of the use of continuous assessment in the teaching of physiology , 1992 .

[25]  Michael T. Kane,et al.  An argument-based approach to validity. , 1992 .

[26]  Yves Benett,et al.  The Validity and Reliability of Assessments and Self‐assessments of Work‐based Learning , 1993 .

[27]  Robert L. Linn,et al.  Educational Assessment: Expanded Expectations and Challenges , 1993 .

[28]  Robert J. Mislevy,et al.  Test Theory for A New Generation of Tests , 1994 .

[29]  S. Messick Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[30]  Direct Assessment, Direct Validation? An Example From the Assessment of Writing , 1994 .

[31]  David J. Bateson Psychometric and Philosophic Problems in "Authentic" Assessment: Performance Tasks and Portfolios. , 1994 .

[32]  Menucha Birenbaum,et al.  Toward adaptive assessment — The student's angle , 1994 .

[33]  S. Messick The Interplay of Evidence and Consequences in the Validation of Performance Assessments , 1994 .

[34]  Michael Prosser,et al.  Students' experiences in studying for multiple choice question examinations , 1994 .

[35]  N. Falchikov Peer Feedback Marking: Developing Peer Assessment , 1995 .

[36]  D. Boud Assessment and learning: contradictory or complementary? , 1995 .

[37]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[38]  Keith Pond,et al.  Peer Review: a Precursor to Peer Assessment , 1995 .

[39]  Liz McDowell,et al.  The Impact of Innovative Assessment on Student Learning , 1995 .

[40]  David Nevo,et al.  School-based evaluation : a dialogue for school improvement , 1995 .

[41]  P. Black Curriculum and assessment in science education: the policy interface , 1995 .

[42]  Hoi K. Suen,et al.  Parent-Professional Congruence , 1995 .

[43]  Filip Dochy,et al.  Integrating assessment, learning and instruction: Assessment of domain-specific and domaintranscending prior knowledge and progress , 1996 .

[44]  Menucha Birenbaum,et al.  Alternatives in Assessment of Achievements, Learning Processes, and Prior Knowledge , 1996 .

[45]  J. Biggs Assessing Learning Quality: reconciling institutional, staff and educational demands , 1996 .

[46]  Edward H. Haertel,et al.  Generalizability Analysis for Performance Assessments of Student Achievement or School Effectiveness , 1997 .

[47]  Filip Dochy,et al.  Assessment and feedback as student support devices , 1997 .

[48]  F. Dochy,et al.  Assessment as a tool for learning , 1997 .

[49]  L. McDowell,et al.  "BUT IS IT FAIR?" : AN EXPLORATORY STUDY OF STUDENT PERCEPTIONS OF THE CONSEQUENTIAL VALIDITY OF ASSESSMENT , 1997 .

[50]  Phil Askham An instrumental response to the instrumental student: Assessment for learning , 1997 .

[51]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[52]  Instructietechnologie en innovatie van probleemoplossen : over constructiegericht academisch onderwijs , 1998 .

[53]  Nancy Falchikov,et al.  “Full on Until the Sun Comes Out”: the effects of assessment on student approaches to studying , 1998 .

[54]  Peter Knight,et al.  Assessment for learning in higher education , 1998 .

[55]  K. Topping Peer Assessment Between Students in Colleges and Universities , 1998 .

[56]  K. Scouller The influence of assessment method on students' learning approaches: Multiple choice question examination versus assignment essay , 1998 .

[57]  A. Baratta,et al.  Guest editor preface , 1998 .

[58]  Karen Sheingold,et al.  Reasoning About Evidence in Portfolios: Cognitive Foundations for Valid and Reliable Assessment , 1998 .

[59]  J. Biggs Assessment and Classroom Learning: a role for summative assessment? , 1998 .

[60]  De toets als hefboom voor meer en beter leren , 1998 .

[61]  D. Sluijsmans,et al.  The use of self-, peer and co-assessment in higher education: A review , 1999 .

[62]  Sally Brown,et al.  Assessment Matters in Higher Education: Choosing and Using Diverse Approaches. , 1999 .

[63]  Xitao Fan,et al.  Published Studies of Interrater Reliability Often Overestimate Reliability: Computing the Correct Coefficient , 2000 .

[64]  P. Orsmond,et al.  The Use of Student Derived Marking Criteria in Peer and Self-assessment , 2000 .

[65]  W. Firestone,et al.  Rethinking "High Stakes": Lessons from the United States and England and Wales. , 2000 .

[66]  Filip Dochy,et al.  A new assessment era: different needs, new challenges , 2001 .

[67]  M. Segers,et al.  Quality standards for new modes modes of assessment. An exploratory study of the consequential validity of the OverAll Test , 2001 .

[68]  Assessment in het hoger onderwijs , 2001 .

[69]  Filip Dochy,et al.  New Lines in Edumetrics: New Forms of Assessment Lead to New Assessment Criteria. , 2001 .

[70]  Mien Segers,et al.  New Assessment Forms in Problem-based Learning: The value-added of the students' perspective , 2001 .

[71]  Stephen E. Newstead,et al.  The Pressures of Assessment in Undergraduate Courses and their Effect on Student Behaviours , 2001 .

[72]  W. Lens,et al.  The Evaluation Practice of Teachers as a Learning Opportunity for Students. , 2001 .

[73]  Nigel Kenneth Pope,et al.  An Examination of the Use of Peer Rating for Formative Assessment in the Context of the Theory of Consumption Values , 2001 .

[74]  Assessment in het hoger onderwijs: over de implicaties van nieuwe toetsvormen voor de edumetrie , 2001 .

[75]  Assessment in onderwijs : nieuwe toetsvormen en examinering in studentgericht onderwijs en competentiegericht onderwijs , 2002 .

[76]  David C. Berliner,et al.  High-Stakes Testing & Student Learning , 2002 .

[77]  Dominique Sluijsmans,et al.  Student involvement in assessment. The training of peer assessment skills. , 2002 .

[78]  W. Harlen,et al.  Testing and Motivation for Learning , 2003 .

[79]  Mien Segers,et al.  Optimising new modes of assessment : in search of qualities and standards , 2003 .

[80]  Sarah Gielen,et al.  Evaluating the Consequential Validity of New Modes of Assessment: The Influence of Assessment on Learning, Including Pre-, Post-, and True Assessment Effects , 2003 .

[81]  Royce Sadler Book Review. Segers, M., Dochy, F. & Cascallar, E. (Eds). (2003). Optimising new modes of assessment: In search of qualities and standards. Dordrecht, Netherlands: Kluwer. , 2004 .

[82]  Students' perceptions on new modes of assessment and their influence on student learning: the portfolio case , 2004 .

[83]  Diane Dancer,et al.  Student involvement in assessment: a project designed to assess class participation fairly and reliably , 2005 .

[84]  Nigel K. L. Pope,et al.  The impact of stress in self‐ and peer assessment , 2005 .

[85]  Mien Segers,et al.  Influence of Redesigning a Learning Environment on Student Perceptions and Learning Strategies , 2005 .

[86]  S. B. Nolen,et al.  Uses and Abuses of Achievement Test Scores. , 2005 .

[87]  Paul A. Kirschner,et al.  The wheel of competency assessment , 2005 .

[88]  Trainees' perspectives on the assessment and self‐assessment of surgical skills , 2005 .

[89]  W. Rod Cullen,et al.  Peer assessment of oral presentations: effects of student gender, university affiliation and participation in the development of assessment criteria , 2005 .

[90]  F. Dochy,et al.  Students’ perceptions about evaluation and assessment in higher education: a review1 , 2005 .

[91]  Robert L. Brennan,et al.  Generalizability of Performance Assessments , 2005 .

[92]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[93]  P. Kirschner,et al.  Teachers ’ opinions on quality criteria for Competency Assessment Programmes , 2009 .

[94]  Velda McCune,et al.  Instructional Psychology: Past, present and future trends , 2006 .

[95]  M. Segers,et al.  Assessment engineering: breaking down barriers between teaching and learning, and assessment , 2007 .

[96]  Patrick Onghena,et al.  A comparative study of peer and teacher feedback and of various peer feedback forms in a secondary school writing curriculum , 2010 .