A mutilevel decomposition of school performance using robust nonparametric frontier techniques ∗

espanolEste documento de trabajo evalua el desempeno educativo desde una optica multinivel, utilizando fronteras parciales. Se consideran variables idiosincraticas a nivel de escuela, clase y alumno. El estudio se ha aplicado a estudiantes de cuarto ano de educacion basica en Chile. De acuerdo con los resultados, menos del 30% de la variabilidad del rendimiento academico de los estudiantes puede ser atribuido a sus escuelas. Se corrobora tambien que un modelo formado unicamente por variables a nivel de estudiante daria lugar a elevadas ineficiencias no atribuibles a la gestion de la escuela, sino mas bien a una inadecuada asignacion de recursos; esto es, si se ignoran las variables especificas de recursos asignados a las escuelas, su desempeno es infravalorado, pues las ineficiencias causadas por una asignacion de recursos suboptima o las dificultades debidas a un entorno socioeconomico desfavorable se atribuyen a una gestion deficiente del centro. EnglishThis working paper proposes a methodology for evaluating educational performance, from a multilevel perspective. We use partial frontier approaches to mitigate the influence of outliers and the curse of dimensionality. We consider idiosyncratic variables at the school, class, and student levels. Our model is applied to a sample of students in fourth year of primary school in Chile. Results are consistent with previous ones that found that less than 30% of the variance in students’ educational attainment was attributable to schools. Our application also corroborates that a model considering only student-level variables would yield high inefficiencies not attributable to the school management, but rather to inadequate resource-endowments. In other words, when disregarding specific variables concerning the resources allocated to schools, the performance of those schools is undervalued, largely because inefficiencies caused by suboptimal resource endowments or difficulties that arise from the socioeconomic environment are instead attributed to poor management

[1]  Abraham Charnes,et al.  Measuring the efficiency of decision making units , 1978 .

[2]  Carlos Santos,et al.  Assessing school efficiency in Portugal using FDH and bootstrapping , 2005 .

[3]  S. Bretschneider,et al.  Estimating school efficiency: A comparison of methods using simulated data , 2001 .

[4]  M. Aitkin,et al.  Statistical Modelling Issues in School Effectiveness Studies , 1986 .

[5]  María Jesús Mancebón,et al.  Performance in primary schools , 2000, J. Oper. Res. Soc..

[6]  Dan Goldhaber,et al.  Methodological Thoughts on Measuring the Impact of Private Sector Competition on the Educational Marketplace , 2003 .

[7]  Alejandra Mizala,et al.  Factores socioeconómicos explicativos de los resultados escolares en la educación secundaria en Chile , 2001 .

[8]  Emili Tortosa-Ausina,et al.  A bootstrap approach to analyse productivity growth in European banking , 2010, J. Oper. Res. Soc..

[9]  Qi Li Nonparametric testing of closeness between two unknown distribution functions , 1996 .

[10]  B. Park,et al.  A NOTE ON THE CONVERGENCE OF NONPARAMETRIC DEA ESTIMATORS FOR PRODUCTION EFFICIENCY SCORES , 1998, Econometric Theory.

[11]  L. Weill Measuring Cost Efficiency in European Banking: A Comparison of Frontier Techniques , 2004 .

[12]  P. W. Wilson,et al.  ASYMPTOTICS AND CONSISTENT BOOTSTRAPS FOR DEA ESTIMATORS IN NONPARAMETRIC FRONTIER MODELS , 2008, Econometric Theory.

[13]  Gary D. Ferrier,et al.  Measuring cost efficiency in banking: Econometric and linear programming evidence , 1990 .

[14]  J. Cummins,et al.  Comparison of Frontier Efficiency Methods: An Application to the U.S. Life Insurance Industry , 1998 .

[15]  Pam Sammons,et al.  Forging Links: Effective Schools and Effective Departments , 1997 .

[16]  Yanqin Fan,et al.  Semiparametric Estimation of Stochastic Production Frontier Models , 1996 .

[17]  M. Urquiola,et al.  Socioeconomic status or noise? Tradeoffs in the generation of school quality information , 2007 .

[18]  Léopold Simar,et al.  Detecting Outliers in Frontier Models: A Simple Approach , 2003 .

[19]  Harold O. Fried,et al.  The Measurement of Productive Efficiency and Productivity Growth , 2008 .

[20]  P. McEwan The Effectiveness of Public, Catholic, and Non-Religious Private Schools in Chile's Voucher System , 2001 .

[21]  Peer group effects and educational production functions , 1978 .

[22]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[23]  G. Battese,et al.  A Metafrontier Production Function for Estimation of Technical Efficiencies and Technology Gaps for Firms Operating Under Different Technologies , 2004 .

[24]  Dominique Deprins,et al.  Measuring Labor-Efficiency in Post Offices , 2006 .

[25]  C. Loader Local Likelihood Density Estimation , 1996 .

[26]  Mark Schneider,et al.  School choice in Chile: Is it class or the classroom? , 2006 .

[27]  George E. Battese,et al.  Technology Gap, Efficiency, and a Stochastic Metafrontier Function , 2002 .

[28]  B. Park,et al.  THE FDH ESTIMATOR FOR PRODUCTIVITY EFFICIENCY SCORES , 2000, Econometric Theory.

[29]  Emmanuel Thanssoulis,et al.  Setting Achievement Targets for School Children , 1999 .

[30]  Marie-Christine Opdenakker,et al.  Relationship between School Composition and Characteristics of School Process and their Effect on Mathematics Achievement , 2001 .

[31]  Harvey Goldstein,et al.  Methods in School Effectiveness Research , 1997 .

[32]  Francisco Gallego COMPETENCIA Y RESULTADOS EDUCATIVOS: TEORIA Y EVIDENCIA PARA CHILE , 2002 .

[33]  Lawrence M. Seiford,et al.  Data envelopment analysis (DEA) - Thirty years on , 2009, Eur. J. Oper. Res..

[34]  J. Florens,et al.  Nonparametric frontier estimation: a robust approach , 2002 .

[35]  Alejandra Mizala,et al.  Bringing the Schools Back in: The Stratification of Educational Achievement in the Chilean Voucher System. , 2012 .

[36]  B. Bass,et al.  Multifactor Leadership Questionnaire , 2011 .

[37]  M. Phillips What Makes Schools Effective? A Comparison of the Relationships of Communitarian Climate and Academic Climate to Mathematics Achievement and Attendance During Middle School , 1997 .

[38]  R. Cervini Class, school, municipal, and state effects on mathematics achievement in Argentina: a multilevel analysis , 2009 .

[39]  K. Kerstens,et al.  Cost efficiency of Belgian local governments: A comparative analysis of FDH, DEA, and econometric approaches , 1996 .

[40]  D. Winkler Educational Achievement and School Peer Group Composition , 1975 .

[41]  Emili Tortosa‐Ausina,et al.  Desempeño de los centros educativos: ¿un problema de recursos o capacidades organizativas? , 2011 .

[42]  P. McEwan,et al.  How Effective Are Private Schools in Latin America? , 2001, Comparative Education Review.

[43]  John Ruggiero,et al.  On the Measurement and Causes of Technical Inefficiency in Local Public Services: With an Application to Public Education , 1995 .

[44]  S. Waslander,et al.  MARKETS IN EDUCATION: AN ANALYTICAL REVIEW OF EMPIRICAL RESEARCH ON MARKET MECHANISMS IN EDUCATION , 2010 .

[45]  Léopold Simar,et al.  Advanced Robust and Nonparametric Methods in Efficiency Analysis: Methodology and Applications , 2007 .

[46]  A. Repetto,et al.  Using school scholarships to estimate the effect of private education on the academic achievement of low-income students in Chile , 2009 .

[47]  Gary D. Ferrier,et al.  Consistency Conditions for Regulatory Analysis of Financial Institutions : A Comparison of Frontier Efficiency Methods , 1997 .

[48]  M. J. Mancebón,et al.  Private versus public high schools in Spain: disentangling managerial and programme efficiencies , 2008, J. Oper. Res. Soc..

[49]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[50]  H. Patrinos Socioeconomic background, schooling, experience, ability and monetary rewards in Greece , 1995 .

[51]  Alejandra Mizala,et al.  The technical efficiency of schools in Chile , 2002 .

[52]  F. Javier Murillo ¿QUIÉNES SON LOS RESPONSABLES DE LOS RESULTADOS DE LAS EVALUACIONES?: HACIA UN PLANTEAMIENTO DE VALOR AGREGADO EN EDUCACIÓN , 2010 .

[53]  Ali Emrouznejad,et al.  Evaluation of research in efficiency and productivity: A survey and analysis of the first 30 years , 2008 .

[54]  R. R. Russell,et al.  Technological Change, Technological Catch-up, and Capital Deepening: Relative Contributions to Growth and Convergence , 2002 .

[55]  Harvey Goldstein,et al.  School Effectiveness Research and Educational Policy , 2000 .

[56]  Harold O. Fried,et al.  The measurement of productive efficiency : techniques and applications , 1993 .

[57]  Bernardo Lara,et al.  The Effectiveness of Private Voucher Education , 2011 .

[58]  Chang-Tai Hsieh,et al.  The effects of generalized school choice on achievement and stratification: Evidence from Chile's voucher program , 2006 .

[59]  David C. Wheelock,et al.  Robust Non-Parametric Quantile Estimation of Efficiency and Productivity Change in U.S. Commercial Banking, 1985-2004 , 2007 .

[60]  School Outcomes: Sharing the Responsibility Between Pupil and School1 , 2002 .

[61]  M. Carnoy,et al.  The Effectiveness and Efficiency of Private Schools in Chile's Voucher System , 2000 .

[62]  Lung-fei Lee Generalized Econometric Models with Selectivity , 1983 .

[63]  P. W. Wilson,et al.  Estimation and Inference in Nonparametric Frontier Models: Recent Developments and Perspectives , 2013 .

[64]  Jeff Kennington,et al.  An Application of Mathematical Programming to Assess Productivity in the Houston Independent School District , 1982 .

[65]  Diego Prior,et al.  Devolution Dynamics of Spanish Local Government , 2010 .

[66]  Bernardita Vial,et al.  THE PERFORMANCE OF PRIVATE AND PUBLIC SCHOOLS IN THE CHILEAN VOUCHER SYSTEM , 2002 .

[67]  P. McEwan Peer effects on student achievement: evidence from Chile , 2003 .

[68]  Emili Tortosa‐Ausina,et al.  Social Capital and Bank Performance: An International Comparison for OECD Countries , 2008 .

[69]  J. Heckman Sample selection bias as a specification error , 1979 .

[70]  P. W. Wilson,et al.  Statistical Inference in Nonparametric Frontier Models: The State of the Art , 1999 .

[71]  Anita A. Summers,et al.  Do Schools Make a Difference , 1977 .

[72]  Emmanuel Thanassoulis,et al.  Assessing pupil and school performance by non-parametric and parametric techniques , 2010, J. Oper. Res. Soc..

[73]  A. Mizala,et al.  School and teacher performance incentives: The Latin American experience , 2004 .

[74]  Carl L. Bankston,et al.  Effect of School Population Socioeconomic Status on Individual Academic Achievement , 1997 .

[75]  Emmanuel Thanassoulis,et al.  Decomposing school and school-type efficiency , 2001, Eur. J. Oper. Res..

[76]  G. Battese,et al.  Metafrontier frameworks for the study of firm-level efficiencies and technology ratios , 2008 .

[77]  Andrea Resti Evaluating the cost-efficiency of the Italian Banking System: What can be learned from the joint application of parametric and non-parametric techniques , 1997 .

[78]  P. Ouellette,et al.  An evaluation of the efficiency of Québec's school boards using the Data Envelopment Analysis method , 2005 .

[79]  H. Goldstein Multilevel Statistical Models , 2006 .

[80]  Francisco A. Gallego Voucher-School Competition , Incentives , and Outcomes : Evidence from Chile ∗ , 2005 .

[81]  S. Kumbhakar,et al.  When, where and how to perform efficiency estimation , 2011, SSRN Electronic Journal.

[82]  Ana S. Camanho,et al.  Analysis of complementary methodologies for the estimation of school value added , 2010, J. Oper. Res. Soc..

[83]  Jesús T. Pastor,et al.  Chapter 3 Translation invariance in data envelopment analysis: A generalization , 1996, Ann. Oper. Res..