Scenario-based assessment of learning experiences

Learning assessment continues to challenge instructors in higher education. Particularly challenging is finding assessment methods that provide insights into both how well a particular learning activity performs, and what can be done to enhance its effectiveness. This paper presents a method for assessing the learning that derives from course-related experiences, both in and out of the classroom. The method uses students' self-reported learning scenarios as the focal unit of analysis. Students are asked to write two scenarios each semester. One scenario describes a valuable learning experience, and the other a situation where they felt a course-related learning activity was not effective. Results from use of the method suggest it may be a generally effective tool for both assessment and design/re-design of learning activity. Effectiveness of the method is supported by evidence from an object-oriented design and programming course over four semesters.

[1]  Dennis R. Honabach.  Responding to 'Educating Lawyers': An Heretical Essay in Support of Abolishing Teaching Evaluations , 2008 .

[2]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[3]  Thomas A. Angelo,et al.  Classroom Assessment Techniques: A Handbook for College Teachers. Second Edition. , 1993 .

[4]  Sandeep Purao,et al.  Situating evaluation in scenarios of use , 2004, CSCW.

[5]  D. Price Jones,et al.  The Year 2000: A Framework for Speculation on the next Thirty-three Years. , 1969 .

[6]  Mary Beth Rosson,et al.  Getting around the task-artifact cycle: how to make claims and design by scenario , 1992, TOIS.

[7]  M. Scriven.  The methodology of evaluation , 1966 .

[8]  N. Granott,et al.  Microdevelopment: Transition Processes in Development and Learning , 2009 .

[9]  Robert Sproule,et al.  The student evaluation of teaching: its failure as a research program, and as an administrative guide , 2009 .

[10]  John M. Carroll,et al.  Making Use: Scenario-Based Design of Human-Computer Interactions , 2000 .

[11]  Robert McCartney,et al.  Program assessment tools in computer science: a report from the trenches , 2003 .

[12]  Herbert W. Marsh,et al.  Do university teachers become more effective with experience? A multilevel growth model of students' evaluations of teaching over 13 years. , 2007 .

[13]  P. Abrami,et al.  Students' Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research , 1987 .

[14]  E. Hargreaves.  Assessment for learning? Thinking outside the (black) box , 2005 .

[15]  Dennis E. Clayson,et al.  Student Evaluations of Teaching: Are They Related to What Students Learn? , 2009 .

[16]  W.J. Staats,et al.  Enhancing an object-oriented curriculum: metacognitive assessment and training , 1999, FIE'99 Frontiers in Education. 29th Annual Frontiers in Education Conference. Designing the Future of Science and Engineering Education. Conference Proceedings (IEEE Cat. No.99CH37011.