Bayesian Modeling of School Effects Using Hierarchical Models with Smoothing Priors

We describe a new and flexible framework for modeling school effects. Like previous work in this area, we introduce an empirical model that evaluates school performance on the basis of student level test-score gains. Unlike previous work, however, we introduce a flexible model that relates follow-up student test scores to baseline student test scores and explore for possible nonlinearities in these relationships.Using data from High School and Beyond (HSB) and adapting the methodology described in Koop and Poirier (2004a), we test and reject the use of specifications that have been frequently used in research and as a basis for policy. We find that nonlinearities are important in the relationship between intake and follow-up achievement, that rankings of schools are sensitive to the model employed, and importantly, that commonly used specifications can give different and potentially misleading assessments of school performance. When estimating our preferred semiparametric specification, we find small but ``significant'' impacts of some school quality proxies (such as district-level expenditure per pupil) in the production of student achievement.

[1]  D. Lindley,et al.  Bayes Estimates for the Linear Model , 1972 .

[2]  Thomas J. Kane,et al.  The Promise and Pitfalls of Using Imprecise School Accountability Measures , 2002 .

[3]  M. Aitkin,et al.  Statistical Modelling Issues in School Effectiveness Studies , 1986 .

[4]  H. Goldstein,et al.  Using Examination Results as Indicators of School and College Performance , 1996 .

[5]  D. Poirier,et al.  Bayesian Variants of Some Classical Semiparametric Regression Techniques , 2004 .

[6]  V. Lee,et al.  Effects of High School Restructuring and Size on Early Gains in Achievement and Engagement , 1995 .

[7]  A. Rukhin Bayes and Empirical Bayes Methods for Data Analysis , 1997 .

[8]  Bradley P. Carlin,et al.  BAYES AND EMPIRICAL BAYES METHODS FOR DATA ANALYSIS , 1996, Stat. Comput..

[9]  Adrian E. Raftery,et al.  Hypothesis testing and model selection , 1996 .

[10]  H. Goldstein,et al.  Multivariate multilevel analyses of examination results , 2002 .

[11]  Empirical Bayesian inference in a nonparametric regression model , 2005 .

[12]  J. Willms Catholic-School Effects on Academic Achievement: New Evidence from the High School and Beyond Follow-Up Study. , 1985 .

[13]  Harvey Goldstein,et al.  League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance , 1996 .

[14]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[15]  J. Coleman,et al.  Achievement Growth in Public and Catholic Schools. , 1985 .

[16]  J. Mulligan,et al.  Classmates' Effects on Black Student Achievement in Public School Classrooms. , 1991 .

[17]  Julian R. Betts,et al.  Does School Quality Matter? Evidence from the National Longitudinal Survey of Youth , 1995 .

[18]  E. Hanushek,et al.  Teachers, Schools, and Academic Achievement , 1998 .