Reframing conversations about teacher quality: school and district administrators’ perceptions of the validity, reliability, and justifiability of a new teacher evaluation system

In changing accountability contexts, policymakers are engaging in international dialogue and collaborative efforts with new opportunities to reframe conversations about how to measure teacher quality and to (re) design and implement evaluation systems accordingly to ensure that they are fair, useable, feasible, and accurate. This study examined the lived experiences of school and district administrators in a large, fast-growth, suburban district in the USA regarding their districts’ new teacher evaluation system to better understand their perceptions of the system’s validity and reliability such that justifiable conclusions may be drawn about teachers’ effectiveness. Given concerns regarding validity and reliability, administrators generally discouraged external, high-stakes uses of evaluation results but valued the evaluation process and the data it provides for supporting teacher growth. As part of a larger study including teachers, findings can inform policymakers seeking to reform teacher evaluation frameworks to emphasize professional growth over high-stakes consequences.

[1]  Matthew A. Kraft,et al.  Revisiting The Widget Effect: Teacher Evaluation Reforms and the Distribution of Teacher Effectiveness , 2017 .

[2]  R. Sitgreaves.  Psychometric theory (2nd ed.). , 1979 .

[3]  A. Strauss,et al.  Basics of qualitative research: Grounded theory procedures and techniques. , 1993 .

[4]  S. Messick.  THE STANDARD PROBLEM: MEANING AND VALUES IN MEASUREMENT AND EVALUATION , 1974 .

[5]  A. Fink.  Evaluation for Education and Psychology , 1995 .

[6]  Sandy Taut,et al.  The Development and Implementation of a National, Standards-based, Multi-method Teacher Performance Assessment System in Chile , 2014 .

[7]  A. Amrein-Beardsley,et al.  Putting Growth and Value-Added Models on the Map: A National Overview , 2014, Teachers College Record: The Voice of Scholarship in Education.

[8]  James H. Stronge,et al.  Linking Teacher Evaluation And Student Learning , 2000 .

[9]  S. Messick.  Test validity and the ethics of assessment. , 1980 .

[10]  D. B. Reid.  School principals acting as middle leaders implementing new teacher evaluation systems , 2018, School Leadership & Management.

[11]  Teachers’ and school administrators’ attitudes and beliefs of teacher evaluation: a preliminary investigation of high poverty school districts , 2018 .

[12]  Derek C. Briggs,et al.  Due Diligence and the Evaluation of Teachers: A Review of the Value-Added Analysis Underlying the Effectiveness Rankings of Los Angeles Unified School District Teachers by the "Los Angeles Times". , 2011 .

[13]  H. Hazi.  VAM Under Scrutiny: Teacher Evaluation Litigation in the States , 2017 .

[14]  J. Papay.  Different Tests, Different Answers , 2011 .

[15]  J. Wyckoff,et al.  Working Paper : Teacher Turnover , Teacher Quality , and Student Achievement in DCPS , 2016 .

[16]  L. Darling-Hammond.  Getting Teacher Evaluation Right: What Really Matters for Effectiveness and Improvement , 2013 .

[17]  D. Berliner.  Between Scylla and Charybdis: Reflections on and problems associated with the evaluation of teachers in an era of metrification , 2018 .

[18]  M. Kraft,et al.  Can Principals Promote Teacher Development as Evaluators? A Case Study of Principals’ Views and Experiences , 2016, Educational administration quarterly : EAQ.

[19]  M. Kane.  Terminology, Emphasis, and Utility in Validation , 2008 .

[20]  S. Al-Samarrai,et al.  World Development Report 2018 : Learning to Realize Education's Promise , 2017 .

[21]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[22]  Jason A. Grissom,et al.  Make Room Value Added , 2015 .

[23]  Theresa Alviar-Martin,et al.  “There is Space, and There are Limits”: The Challenge of Teaching Controversial Topics in an Illiberal Democracy , 2014, Teachers College Record: The Voice of Scholarship in Education.

[24]  Douglas N. Harris,et al.  Value-Added Measures in Education: What Every Educator Needs to Know , 2011 .

[25]  Mary Lee Smith.  Mixing and matching: Methods and models , 1997 .

[26]  L. Darling-Hammond.  Can Value Added Add Value to Teacher Evaluation? , 2015 .

[27]  M. Derrington,et al.  High-stakes teacher evaluation policy: US principals’ perspectives and variations in practice , 2018 .

[28]  R. Stake.  The Case Study Method in Social Inquiry1 , 1978 .

[29]  M. Derrington,et al.  School principals’ views of teacher evaluation policy: lessons learned from two empirical studies , 2017 .

[30]  Noelle A. Paufler.  Declining Morale, Diminishing Autonomy, and Decreasing Value: Principal Reflections on a High-Stakes Teacher Evaluation System , 2018, International Journal of Education Policy and Leadership.

[31]  G. Schumacher,et al.  New teachers’ perceptions of a standards-based performance appraisal system , 2012 .

[32]  Linda Darling-Hammond,et al.  Evaluating Teacher Evaluation , 2012 .

[33]  Jason A. Grissom,et al.  Using Teacher Effectiveness Data for Information-Rich Hiring , 2017 .

[34]  Matthew G. Springer,et al.  Using Student Test Scores to Measure Teacher Performance , 2015 .

[35]  M. Derrington,et al.  Teacher Evaluation Policy Tools: Principals’ Selective Use in Instructional Leadership , 2018 .

[36]  H. Hendry,et al.  Teachers’ views of the impact of school evaluation and external inspection processes , 2016 .

[37]  Roger Chamberlain,et al.  Teacher evaluation in Illinois: school leaders’ perceptions and practices , 2017 .

[38]  R. Linn,et al.  Qualitative methods in research on teaching , 1985 .

[39]  Morgaen L. Donaldson.  Principals' Approaches to Developing Teacher Quality: Constraints and Opportunities in Hiring, Assigning, Evaluating, and Developing Teachers. , 2011 .

[40]  Sandy Taut,et al.  Classroom observation for evaluating and improving teaching: An international perspective , 2016 .

[41]  Matthew Finster,et al.  Teacher perceptions of a new performance evaluation system and their influence on practice: A within- and between-school level analysis , 2018 .

[42]  M. Simone.  Already Connected Digital Early Childhood between Digital Media and the Net , 2020, International Journal of Education.

[43]  Alyson L. Lavigne.  Exploring the Intended and Unintended Consequences of High-Stakes Teacher Evaluation on Schools, Teachers, and Students , 2014, Teachers College Record: The Voice of Scholarship in Education.

[44]  A. Amrein-Beardsley.  Rethinking Value-Added Models in Education: Critical Perspectives on Tests and Assessment-Based Accountability , 2014 .

[45]  Johnny Saldaña,et al.  The Coding Manual for Qualitative Researchers , 2009 .

[46]  Douglas N. Harris,et al.  How Teacher Evaluation Methods Matter for Accountability , 2014 .

[47]  Audrey Amrein-Beardsley,et al.  The SAS Education Value-Added Assessment System (SAS® EVAAS®) in the Houston Independent School District (HISD): Intended and Unintended Consequences , 2012 .

[48]  B MilesMatthew,et al.  Qualitative Data Analysis , 2018, Approaches and Processes of Social Science Research.

[49]  Susan E. Sporte,et al.  Teacher Perspectives on Evaluation Reform , 2015 .

[50]  David Keeling,et al.  The Widget Effect: Our National Failure to Acknowledge and Act on Differences in Teacher Effectiveness. Second Edition. , 2009 .

[51]  Heather C. Hill,et al.  State and Local Efforts to Investigate the Validity and Reliability of Scores from Teacher Evaluation Systems , 2014, Teachers College Record: The Voice of Scholarship in Education.

[52]  N. Ntoumanis,et al.  Does teacher evaluation based on student performance predict motivation, well-being, and ill-being? , 2018, Journal of school psychology.

[53]  A. Amrein-Beardsley.  Methodological Concerns About the Education Value-Added Assessment System , 2008 .

[54]  Shujie Liu,et al.  Teacher evaluation in China: latest trends and future directions , 2013 .

[55]  M. Derrington.  Implementing Teacher Evaluation , 2016 .

[56]  P. Hopkins.  Teacher Voice , 2016 .