Does Teaching Quality Cross Subjects? Exploring Consistency in Elementary Teacher Practice Across Subjects

Teacher evaluation systems treat instructional quality as generic. Principals often observe elementary teachers in one subject and generalize assessments to all subjects. However, there is little empirical work to justify these decisions. This study provides needed evidence about whether elementary teachers engage in comparable instruction across the school day. We draw on data from the Measures of Effective Teaching (MET) project, including student survey and classroom observational data from more than 500 elementary teachers. Findings indicate that there is moderate within-teacher, cross-subject consistency on the Tripod and Classroom Observation Scoring System (CLASS). Cross-subject correlations are higher on the Tripod scales (r values from 0.55 to 0.73) than the CLASS dimensions and domains (r values from 0.25 to 0.55). These findings suggest that teaching quality is not a uniform construct across subjects, even though current teaching evaluation systems largely treat it as such. Implications for elementary teacher preparation, professional development, and evaluation are discussed.

[1]  K. Yoon,et al.  What Works in Professional Development? , 2009 .

[2]  D. Reutzel,et al.  Explicit Instruction in Core Reading Programs , 2014, The Elementary School Journal.

[3]  Daniel F. McCaffrey,et al.  Have We Identified Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment. Research Paper. MET Project. , 2013 .

[4]  Robert C. Pianta,et al.  Teaching through Interactions , 2013, The Elementary School Journal.

[5]  Charalambos Y. Charalambous,et al.  When Rater Reliability Is Not Enough , 2012 .

[6]  S. Stodolsky,et al.  Content as Context: The Role of School Subjects in Secondary School Teaching , 1995 .

[7]  Heather C. Hill,et al.  Learning from Teacher Observations: Challenges and Opportunities Posed by New Teacher Evaluation Systems. , 2013 .

[8]  Marilyn J. Chambliss,et al.  Crossing the Borders Again: Challenges in Comparing Quality Instruction in Mathematics and Reading , 2012 .

[9]  Youlanda M. Gibbons,et al.  Trust in Schools: A Core Resource for Improvement , 2004 .

[10]  Susanna Loeb,et al.  How Stable Are Value-Added Estimates across Years, Subjects and Student Groups? What We Know Series: Value-Added Methods and Applications. Knowledge Brief 3. , 2012 .

[11]  Raymond L. Pecheone,et al.  Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains , 2012 .

[12]  S. Stodolsky.  The Subject Matters: Classroom Activity in Math and Social Studies , 1988 .

[13]  Amanda Berry,et al.  Teacher Professional Development Focusing on Pedagogical Content Knowledge , 2012 .

[14]  Andrew D. Ho,et al.  The Reliability of Classroom Observations by School Personnel. Research Paper. MET Project. , 2013 .

[15]  Frank M. Pajares,et al.  Self-Efficacy Beliefs in Academic Settings , 1996 .

[16]  To What Extent Do Student Perceptions of Classroom Quality Predict Teacher Value Added , 2015 .

[17]  Christine M. Neumerski.  Rethinking Instructional Leadership, a Review , 2013 .

[18]  Brian E. Clauser,et al.  An Examination of Rater Drift within a Generalizability Theory Framework. , 2009 .

[19]  Drew H. Gitomer,et al.  Measurement Issues and Assessment for Teaching Quality , 2008 .

[20]  Dan Goldhaber,et al.  Building a More Complete Understanding of Teacher Evaluation Using Classroom Observations , 2016 .

[21]  Practices that cross disciplines?: Revisiting explicit instruction in elementary mathematics and English language arts , 2018 .

[22]  J. Allen,et al.  Observations of Effective Teacher–Student Interactions in Secondary School Classrooms: Predicting Student Achievement With the Classroom Assessment Scoring System—Secondary , 2013, School psychology review.

[23]  Robert C. Pianta,et al.  An Argument Approach to Observation Protocol Validity , 2012 .

[24]  Matthew A. Kraft,et al.  Revisiting The Widget Effect: Teacher Evaluation Reforms and the Distribution of Teacher Effectiveness , 2017 .

[25]  J. Stigler,et al.  Teaching Versus Teachers as a Lever for Change: Comparing a Japanese and a U.S. Perspective on Improving Instruction , 2017 .

[26]  M. Kraft,et al.  Can Professional Environments in Schools Promote Teacher Development? Explaining Heterogeneity in Returns to Teaching Experience , 2014, Educational evaluation and policy analysis.

[27]  Peter A. Youngs,et al.  Teacher Characteristics and Student Achievement Gains: A Review , 2003 .

[28]  Andrew J. Mashburn,et al.  Improving the Power of an Efficacy Study of a Social and Emotional Learning Program: Application of Generalizability Theory to the Measurement of Classroom-Level Outcomes , 2014, Prevention Science.

[29]  Matthew Ronfeldt,et al.  The Test Matters , 2014 .

[30]  Commonality and Uniqueness in Teaching Practice Observation , 2013 .

[31]  J. Schweig.  Cross-Level Measurement Invariance in School and Classroom Environment Surveys , 2014 .

[32]  J. Remillard.  Examining Key Concepts in Research on Teachers’ Use of Mathematics Curricula , 2005 .

[33]  Charalambos Y. Charalambous,et al.  Mathematical Knowledge for Teaching and the Mathematical Quality of Instruction: An Exploratory Study , 2008 .

[34]  S. Raudenbush,et al.  Resources, Instruction, and Research , 2003 .

[35]  Robert C. Pianta,et al.  Classroom Effects on Children’s Achievement Trajectories in Elementary School , 2008 .

[36]  J. Allen,et al.  An Interaction-Based Approach to Enhancing Secondary School Instruction and Student Achievement , 2011, Science.

[37]  Julie Cohen.  Challenges in Identifying High-Leverage Practices , 2015, Teachers College Record: The Voice of Scholarship in Education.

[38]  D. Kagan.  Implication of Research on Teacher Belief , 1992 .

[39]  E. Hirsch,et al.  How Working Conditions Predict Teaching Quality and Student Outcomes , 2015 .

[40]  D. Ball.  The Mathematical Understandings That Prospective Teachers Bring to Teacher Education , 1990, The Elementary School Journal.

[41]  Susan Johnson,et al.  How Context Matters in High-Need Schools: The Effects of Teachers’ Working Conditions on Their Professional Satisfaction and Their Students’ Achievement , 2012, Teachers College Record: The Voice of Scholarship in Education.

[42]  Dan Goldhaber,et al.  Is a good elementary teacher always good? Assessing teacher performance estimates across subjects , 2013 .

[43]  Erik A. Ruzek,et al.  What Can Student Perception Surveys Tell Us About Teaching? Empirically Testing the Underlying Structure of the Tripod Student Perception Survey , 2016 .

[44]  B. Hamre,et al.  Conceptualization, Measurement, and Improvement of Classroom Processes: Standardized Observation Can Leverage Capacity , 2009 .

[45]  Jason A. Grissom,et al.  Make Room Value Added , 2015 .

[46]  P. Cobb,et al.  The Contextual Nature of Teaching: Mathematics and Reading Instruction in One Second-Grade Classroom , 1990, The Elementary School Journal.

[47]  R. Pianta,et al.  Teacher–Child Interactions in Free Choice and Teacher-Directed Activity Settings: Prediction to School Readiness , 2017 .

[48]  Sheila S. Graybeal,et al.  Instructional Practice in Fifth-Grade Math and Social Studies: An Analysis of Teacher's Guides. , 1986 .

[49]  Castl Survey.  The Carnegie Foundation for the Advancement of Teaching , 1910, Nature.

[50]  Jilliam N. Joe,et al.  Scoring Design Decisions: Reliability and the Length and Focus of Classroom Observations , 2015 .

[51]  Andrew J. Mashburn,et al.  Within-Day Variability in the Quality of Classroom Interactions during Third and Fifth Grade , 2011, The Elementary School Journal.

[52]  M. Kraft,et al.  Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement , 2015 .

[53]  Jonah E. Rockoff,et al.  What does certification tell us about teacher effectiveness? Evidence from New York City , 2008 .

[54]  Megan Kuhfeld.  When Students Grade Their Teachers: A Validity Analysis of the Tripod Student Survey , 2017 .

[55]  Alison E. Baroody.  Exploring the contribution of classroom formats on teaching effectiveness and achievement in upper elementary classrooms , 2017 .

[56]  Michael S. Knapp,et al.  Academic Challenge in High-Poverty Classrooms. , 1995 .

[57]  Demetra Kalogrides,et al.  Effective Schools: Teacher Hiring, Assignment, Development, and Retention , 2011, Education Finance and Policy.

[58]  Courtney A. Bell,et al.  The Instructional Challenge in Improving Teaching Quality: Lessons from a Classroom Observation Protocol , 2014, Teachers College Record: The Voice of Scholarship in Education.

[59]  D. Mewborn,et al.  Teachers Content Knowledge, Teacher Education, and their Effects on the Preparation of Elementary Teachers in the United States , 2001 .

[60]  Jing Chen,et al.  Evaluating Efforts to Minimize Rater Bias in Scoring Classroom Observations , 2015 .

[61]  Robert C. Pianta,et al.  Designing Teacher Evaluation Systems: New Guidance from the Measures of Effective Teaching Project , 2014 .

[62]  Skee Smith,et al.  The U.S. Department of Education. , 1979 .

[63]  Timothy W. Curby,et al.  Do emotional support and classroom organization earlier in the year set the stage for higher quality instruction? , 2013, Journal of school psychology.

[64]  Morgan S. Polikoff.  The Stability of Observational and Student Survey Measures of Teaching Effectiveness , 2014, American Journal of Education.

[65]  Ronald F. Ferguson.  Can Student Surveys Measure Teaching Quality? , 2012 .

[66]  J. Downer,et al.  Measuring Effective Teacher-Student Interactions From a Student Perspective , 2015 .

[67]  R. Kanter,et al.  The Challenge of organizational change : how companies experience it and leaders guide it , 1993 .