IMPLEMENTING TEACHER EVALUATION SYSTEMS How Principals Make Sense of Complex Artifacts to Shape Local Instructional Practice 1

This study examines how local school leaders make sense of complex programs designed to evaluate teachers and teaching. New standards-based teacher evaluation policies promise to provide school leaders and teachers with a common framework that can serve as a basis for improving teaching and learning in schools (Danielson & McGreal, 2000; Odden & Kelley, 2002). However, implementation research suggests that the ways in which local actors make sense of and use such policies determines the nature of the changes that actually occur in schools (Desimone, 2002; Spillane, Reiser, & Reimer, 2002). In this paper, case studies of schools in a large school district are used to examine school-level implementation of a standards-based teacher evaluation system. Specifically, we examine the ways in which school and district leaders emphasize and select from the many features of a teacher evaluation framework in the implementation process. We then discuss the ways in which key features of the process were co-opted, ignored, or adapted in accordance with school context, and we point to how the resulting teacher evaluation practices help to create conditions for more substantive conversations about reforming teaching practice. Introduction While it is generally acknowledged that teachers exert great influence over the improvement of student learning (Darling-Hammond & Ball, 1997; Wright, Horn & Sanders, 1997), the role that school leaders play in shaping system capacity for successful teaching

[1]  L. Darling-Hammond,et al.  Teacher Evaluation in the Organizational Context: A Review of the Literature , 1983 .

[2]  G. Madaus,et al.  Charms Talismanic: Testing Teachers for the Improvement of American Education , 1987 .

[3]  K. Peterson Secondary Principals and Instructional Leadership: Complexities in a Diverse Role. , 1989 .

[4]  James W. Smither Performance appraisal : state of the art in practice , 1998 .

[5]  H. Hazi The Teacher Evaluation-Supervision Dilemma: A Case of Entanglements and Irreconcilable Differences. , 1994 .

[6]  Kenneth D. Forbus,et al.  The Roles of Similarity in Transfer: Separating Retrievability From Inferential Soundness , 1993, Cognitive Psychology.

[7]  D. Cohen,et al.  Bridging the Gap Between Standards and Achievement The Imperative for Professional Development in Education , 2002 .

[8]  Marx W. Wartofsky,et al.  Models: Representation And The Scientific Understanding , 1979 .

[9]  R. Stake The art of case study research , 1995 .

[10]  H. Alix Gallagher,et al.  Vaughn Elementary's Innovative Teacher Evaluation System: Are Teacher Evaluation Scores Related to Growth in Student Achievement? , 2004 .

[11]  Roger C. Schank,et al.  SCRIPTS, PLANS, GOALS, AND UNDERSTANDING , 1988 .

[12]  Susan H. Fuhrman,et al.  Rewards and Reform: Creating Educational Incentives That Work. Jossey-Bass Education Series. , 1996 .

[13]  H. Heneman,et al.  Assessment of Teacher Reactions to a Standards-Based Teacher Evaluation System: A Pilot Study* , 2001 .

[14]  James P. Spillane,et al.  Policy Implementation and Cognition: Reframing and Refocusing Implementation Research , 2002 .

[15]  Frances J. Milliken,et al.  Executives’ Perceptual Filters: What They Notice and How They Make Sense , 1988 .

[16]  E. Reed The Ecological Approach to Visual Perception , 1989 .

[17]  Rocky Ross,et al.  Mental models , 2004, SIGA.

[18]  Choong Seng. Loh Teacher evaluation to enhance professional practice , 2003 .

[19]  D. Bowden Teacher Evaluation: A Comprehensive Guide to New Directions and Practices , 2001 .

[20]  Douglas R. Davis,et al.  Issues in Implementing a New Teacher Assessment System in a Large Urban School District: Results of a Qualitative Field Study , 2000 .

[21]  G. Salomon Distributed cognitions : psychological and educational considerations , 1997 .

[22]  B. Fischhoff,et al.  Hindsight ≠ foresight: the effect of outcome knowledge on judgment under uncertainty* , 2003 .

[23]  Organizations , 1992, Restoration & Management Notes.

[24]  C. McGuire,et al.  Office of Educational Research and Improvement , 1999 .

[25]  S. Kirby,et al.  Looking Back over a Decade of Whole-School Reform: The Experience of New American Schools , 2002 .

[26]  Sandra P. Horn,et al.  Teacher and Classroom Context Effects on Student Achievement: Implications for Teacher Evaluation , 1997 .

[27]  Kevin R. Murphy,et al.  Understanding Performance Appraisal: Social, Organizational, and Goal-Based Perspectives , 1995 .

[28]  R. Heck,et al.  Reassessing the Principal's Role in School Effectiveness: A Review of Empirical Research, 1980-1995 , 1996 .

[29]  S. Carey Conceptual Change in Childhood , 1985 .

[30]  K. Weick Educational organizations as loosely coupled systems , 1976, Gestión y Estrategia.

[31]  C. Kelley,et al.  Paying Teachers for What They Know and Do: New and Smarter Compensation Strategies To Improve Schools. Second Edition. , 1996 .

[32]  J. Confrey A Review of the Research on Student Conceptions in Mathematics, Science, and Programming , 1990 .

[33]  J. Greeno THE SITUATIVITY OF KNOWING, LEARNING, AND RESEARCH , 1998 .

[34]  D. Hambrick,et al.  Book Review: Human Resources, Personnel, and Organizational Behavior: The Executive Effect: Concepts and Methods for Studying Top Managers , 1990 .

[35]  Karen S. Loup,et al.  Ten years later: Findings from a replication of a study of teacher evaluation practices in our 100 largest school districts , 1996 .

[36]  K. Peterson Teacher Evaluation: A Comprehensive Guide to New Directions and Practices. Second Edition. , 1995 .

[37]  William F. Brewer,et al.  The Role of Anomalous Data in Knowledge Acquisition: A Theoretical Framework and Implications for Science Instruction , 1993 .

[38]  J. Garbarino The ecology of human development: Experiments by nature and design: by Urie Bronfenbrenner Cambridge, Mass.: Harvard University Press, 1979, 330 + p. , 1980 .

[39]  K. Weick FROM SENSEMAKING IN ORGANIZATIONS , 2021, The New Economic Sociology.

[40]  Andrew Elby,et al.  On the Form of a Personal Epistemology , 2002 .

[41]  S. Kiesler,et al.  Managerial Response to Changing Environments: Perspectives on Problem Sensing from Social Cognition. , 1982 .

[42]  Karl E. Weick,et al.  Sensemaking in Organizations , 1996 .

[43]  Richard Halverson,et al.  Investigating School Leadership Practice: A Distributed Perspective , 2001 .

[44]  Joseph F. Murphy Transformational Change and the Evolving Role of the Principal: Early Empirical Evidence. , 1994 .

[45]  Laura M. Desimone,et al.  How Can Comprehensive School Reform Models Be Successfully Implemented? , 2002 .

[46]  Charlotte F. Danielson Enhancing Professional Practice: A Framework for Teaching , 1996 .

[47]  S. Klein,et al.  A License to Teach: Raising Standards for Teaching , 1995 .

[48]  Roger C. Schank,et al.  Dynamic memory - a theory of reminding and learning in computers and people , 1983 .

[49]  A. Collins,et al.  Cognition and learning. , 1996 .

[50]  D. Ilgen,et al.  Bearing Bad News: Reactions to Negative Performance Feedback , 2000 .

[51]  L. Darling-Hammond,et al.  Teaching for High Standards: What Policymakers Need To Know and Be Able To Do. CPRE Joint Report Series. , 1998 .

[52]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[53]  Jere Confrey,et al.  Chapter 1: A Review of the Research on Student Conceptions in Mathematics, Science, and Programming , 1990 .

[54]  L. Darling-Hammond,et al.  Teacher Evaluation: A Study of Effective Practices , 1985, The Elementary School Journal.

[55]  B. Ross This is like that: The use of earlier problems and the separation of similarity effects. , 1987 .

[56]  Steven M. Kimball Analysis of Feedback, Enabling Conditions and Fairness Perceptions of Teachers in Three School Districts with New Standards-Based Evaluation Systems , 2002 .

[57]  Donald A. Norman,et al.  Things That Make Us Smart: Defending Human Attributes In The Age Of The Machine , 1993 .

[58]  C SchankRoger,et al.  Dynamic Memory: A Theory of Reminding and Learning in Computers and People , 1983 .

[59]  H. P. Sims,et al.  Behind the Mask: The Politics of Employee Appraisal , 1987 .

[60]  Brian Rowan Commitment and Control: Alternative Strategies for the Organizational Design of Schools , 1990 .

[61]  P. Pintrich,et al.  Personal Epistemology The Psychology of Beliefs About Knowledge and Knowing , 2002 .