Design Experimentation with Multiple Perspectives: The GenScope Assessment Project

The GenScope Assessment Project is studying assessment in the context of a month-long computer-supported learning environment for introductory genetics. Across three annual iterations with multiple teachers, project researchers manipulated the materials, incentives, and contexts in which students were invited to use formative feedback on challenging classroom performance assessments. The consequences of these manipulations on engagement and learning were systematically examined from behavioral/empiricist, cognitive/rationalist, and situative/sociohistoric perspectives. This "comparative approach" was intended to provide new insights into unresolved issues over extrinsic rewards and accountabilityoriented reforms.It turned out that the comparative approach also provided a powerful framework for refining and improving theories about classroom assessment. Essentially researchers "tuned" the classroom assessment environment to maximize gains on carefully aligned external performance assessments. Successive improvements led to correspondingly larger gains on an external achievement test that was more aligned with conventional genetics instruction. This study shows that design-based research around classroom assessment can help meet the wider educational goals of researchers within the increasingly narrow policies of reformers. Five appendixes contain examples of feedback and rubrics. (Contains 10 figures and 80 references.)

[1]  J. Bransford.  How people learn , 2000 .

[2]  L. Schauble,et al.  Design Experiments in Educational Research , 2003 .

[3]  Susan E. Newman,et al.  Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics. Technical Report No. 403. , 1987 .

[4]  J. Shea.  National Science Education Standards , 1995 .

[5]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[6]  Ann C. H. Kindfield.  Understanding a basic biological process: Expert and novice models of meiosis , 1994 .

[7]  J. Frederiksen,et al.  A Systems Approach to Educational Testing , 1989 .

[8]  D. E. Stokes.  Pasteur's Quadrant: Basic Science and Technological Innovation , 1997 .

[9]  Sandra Graham,et al.  Motivation in African Americans , 1994 .

[10]  H. Mandl,et al.  Situated learning and transfer , 1995 .

[11]  J. Osborne,et al.  Establishing the norms of scientific argumentation in classrooms , 2000 .

[12]  Anthony E. Kelly,et al.  Theme Issue: The Role of Design in Educational Research , 2003 .

[13]  Richard J. Shavelson,et al.  Scientific Culture and Educational Research , 2002 .

[14]  M. Graue,et al.  Integrating Theory and Practice Through Instructional Assessment , 1993 .

[15]  Mary M. Kennedy,et al.  Approximations to Indicators of Student Outcomes , 1999 .

[16]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[17]  J. Tudge,et al.  Vygotsky, Piaget, and Bandura: Perspectives on the Relations between the Social World and Cognitive Development. , 1993 .

[18]  Daniel T. Hickey,et al.  Motivation and contemporary socio-constructivist instructional perspectives , 1997 .

[19]  R. Sternberg,et al.  Transfer on Trial: Intelligence, Cognition and Instruction , 1993 .

[20]  A. Collins,et al.  Cognition and learning. , 1996 .

[21]  Daniel T. Hickey,et al.  Educational Psychology, Social Constructivism, and Educational Practice: A Case of Emergent Identity , 2001 .

[22]  S. Messick.  The Interplay of Evidence and Consequences in the Validation of Performance Assessments , 1994 .

[23]  M. Scardamalia,et al.  Intentional Learning as a Goal of Instruction , 2018, Knowing, Learning, and Instruction.

[24]  Mary M. Rohrkemper,et al.  Self-Regulated Learning and Academic Achievement: A Vygotskian View , 1989 .

[25]  Paul Horwitz,et al.  Integrating Instruction, Assessment, & Evaluation in a Technology- Based Genetics Environment: The GenScope™ Follow-up Study , 2000 .

[26]  R. Duschl,et al.  "Doing the Lesson" or "Doing Science": Argument in High School Genetics , 2000 .

[27]  Joel R. Levin,et al.  What to do about educational research's credibility gaps? , 1999 .

[28]  L. Quéré.  Cognition in Practice , 1996 .

[29]  Paul Horwitz,et al.  Advancing Educational Theory by Enhancing Practice in a Technology-Supported Genetics Learning Environment. , 1999 .

[30]  P. Tunstall,et al.  Teacher Feedback to Young Children in Formative Assessment: a typology , 1996 .

[31]  J. Cameron,et al.  Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis , 1994 .

[32]  R. Duschl,et al.  Strategies and Challenges to Changing the Focus of Assessment and Instruction in Science Classrooms. , 1997 .

[33]  David Bloome,et al.  Procedural Display and Classroom Lessons , 1989 .

[34]  Finbarr C. Sloane,et al.  Exploring Modeling Aspects of Design Experiments , 2003 .

[35]  S. Messick.  Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[36]  Abbie Brown,et al.  Design experiments: Theoretical and methodological challenges in creating complex interventions in c , 1992 .