Multidimensional Assessment of Value Added by Teachers to Real-World Outcomes

Measuring teacher effectiveness is challenging since no direct estimate exists; teacher effectiveness can be measured only indirectly through student responses. Traditional value-added assessment (VAA) models generally attempt to estimate the value that an individual teacher adds to students' knowledge as measured by scores on successive administrations of a standardized test. Such responses, however, do not reflect the long-term contribution of a teacher to real-world student outcomes such as graduation, and cannot be used in most university settings where standardized tests are not given. In this paper, the authors develop a multiresponse approach to VAA models that allows responses to be either continuous or categorical. This approach leads to multidimensional estimates of value added by teachers and allows the correlations among those dimensions to be explored. The authors derive sufficient conditions for maximum likelihood estimators to be consistent and asymptotically normally distributed. The authors then demonstrate how to use SAS software to calculate estimates. The models are applied to university data from 2001 to 2008 on calculus instruction and graduation in a science or engineering field.

[1]  Thomas A Louis,et al.  Jump down to Document , 2022 .

[2]  Daniel F. McCaffrey,et al.  Bayesian Methods for Scalable Multivariate Value-Added Assessment , 2007 .

[3]  N. Augustine.  Rising Above The Gathering Storm: Energizing and Employing America for a Brighter Economic Future , 2006 .

[4]  Joseph A. Martineau.  Distorting Value Added: The Use of Longitudinal, Vertically Scaled Student Achievement Data for Growth-Based, Value-Added Accountability , 2006 .

[5]  James Algina,et al.  An Empirical Comparison of Statistical Models for Value-Added Assessment of School Performance , 2004 .

[6]  J. Pinheiro,et al.  Efficient Laplacian and Adaptive Gaussian Quadrature Algorithms for Multilevel Generalized Linear Mixed Models , 2006 .

[7]  C. McCulloch.  Maximum Likelihood Variance Components Estimation for Binary Data , 1994 .

[8]  H. Hartley,et al.  Maximum-likelihood estimation for the mixed analysis of variance model. , 1967, Biometrika.

[9]  Xianglei Chen,et al.  Students Who Study Science, Technology, Engineering, and Mathematics (STEM) in Postsecondary Education. Stats in Brief. NCES 2009-161. , 2009 .

[10]  Paul Wright,et al.  Controlling for Student Background in Value-Added Assessment of Teachers , 2004 .

[11]  J. R. Lockwood,et al.  Fitting Value-Added Models in R , 2006 .

[12]  E. Demidenko,et al.  Mixed Models: Theory and Applications (Wiley Series in Probability and Statistics) , 2004 .

[13]  Stephen W. Raudenbush,et al.  What Are Value-Added Models Estimating and What Does This Imply for Statistical Practice? , 2004 .

[14]  Thomas J. Kane,et al.  Identifying Effective Teachers Using Performance on the Job. The Hamilton Project Policy Brief No. 2006-01. , 2006 .

[15]  N. Goldman,et al.  Improved estimation procedures for multilevel models with binary response: a case‐study , 2001 .

[16]  Daniel F. McCaffrey,et al.  Missing Data in Value-Added Modeling of Teacher Effects , 2011 .

[17]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .

[18]  J. R. Lockwood,et al.  A Model for Teacher Effects From Longitudinal Data Without Assuming Vertical Scaling , 2010 .

[19]  Russ Wolfinger,et al.  Computing Gaussian Likelihoods and Their Derivatives for General Linear Mixed Models , 1994, SIAM J. Sci. Comput..

[20]  R. Wolfinger,et al.  Generalized linear mixed models a pseudo-likelihood approach , 1993 .

[21]  Daniel F. McCaffrey,et al.  Evaluating Value-Added Models for Teacher Accountability , 2004 .

[22]  A. Agresti,et al.  The Authors Replied as Follows: , 2001 .

[23]  H. Goldstein,et al.  Using Examination Results as Indicators of School and College Performance , 1996 .

[24]  Tom A. B. Snijders,et al.  Variance Component Testing in Multilevel Models , 2001 .

[25]  P. Gustafson,et al.  Conservative prior distributions for variance parameters in hierarchical models , 2006 .

[26]  Ana Ivelisse Avilés,et al.  Linear Mixed Models for Longitudinal Data , 2001, Technometrics.

[27]  A. Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[28]  K. Mardia,et al.  Maximum likelihood estimation of models for residual covariance in spatial regression , 1984 .

[29]  E. Lehmann.  Elements of large-sample theory , 1998 .

[30]  K. Liang,et al.  Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests under Nonstandard Conditions , 1987 .

[31]  J. Miller,et al.  Asymptotic Properties of Maximum Likelihood Estimates in the Mixed Model of the Analysis of Variance , 1977 .

[32]  Brian Rowan,et al.  What Large-Scale, Survey Research Tells Us About Teacher What Large-Scale, Survey Research Tells Us About Teacher Effects on Student Achievement: Insights From the Prospectus Effects on Student Achievement: Insights From the Prospectus Study of Elementary Schools Study of Elementary Schools , 2015 .