Efficiency of Health Care Production in Low-Resource Settings: A Monte-Carlo Simulation to Compare the Performance of Data Envelopment Analysis, Stochastic Distance Functions, and an Ensemble Model

Low-resource countries can greatly benefit from even small increases in efficiency of health service provision, supporting a strong case to measure and pursue efficiency improvement in low- and middle-income countries (LMICs). However, the knowledge base concerning efficiency measurement remains scarce for these contexts. This study shows that current estimation approaches may not be well suited to measure technical efficiency in LMICs and offers an alternative approach for efficiency measurement in these settings. We developed a simulation environment which reproduces the characteristics of health service production in LMICs, and evaluated the performance of Data Envelopment Analysis (DEA) and Stochastic Distance Function (SDF) for assessing efficiency. We found that an ensemble approach (ENS) combining efficiency estimates from a restricted version of DEA (rDEA) and restricted SDF (rSDF) is the preferable method across a range of scenarios. This is the first study to analyze efficiency measurement in a simulation setting for LMICs. Our findings aim to heighten the validity and reliability of efficiency analyses in LMICs, and thus inform policy dialogues about improving the efficiency of health service production in these settings.

[1]  Patrick Royston,et al.  The design of simulation studies in medical statistics , 2006, Statistics in medicine.

[2]  J. Akazili,et al.  What are the Technical and Allocative Efficiencies of Public Health Centres in Ghana? , 2008, Ghana medical journal.

[3]  Bryan F. J. Manly,et al.  On the use of correlated beta random variables with animal population modelling , 2008 .

[4]  John Ruggiero,et al.  A comparison of DEA and the stochastic frontier model using panel data , 2007, Int. Trans. Oper. Res..

[5]  Andrew L. Johnson,et al.  A unified model for detecting efficient and inefficient outliers in data envelopment analysis , 2010, Comput. Oper. Res..

[6]  T. Adam,et al.  Exploring the effects of task shifting for HIV through a systems thinking lens: the case of Burkina Faso , 2013, BMC Public Health.

[7]  S. Grosskopf,et al.  Measuring hospital performance. A non-parametric approach. , 1987, Journal of health economics.

[8]  Jim E. Griffin,et al.  Bayesian stochastic frontier analysis using WinBUGS , 2007 .

[9]  Emmanuel Thanassoulis,et al.  Zero weights and non-zero slacks: Different solutions to the same problem , 2006, Ann. Oper. Res..

[10]  John Ruggiero,et al.  A new approach for technical efficiency estimation in multiple output production , 1998, Eur. J. Oper. Res..

[11]  S. Bretschneider,et al.  Estimating school efficiency: A comparison of methods using simulated data , 2001 .

[12]  N. Avkiran Productivity Analysis in the Service Sector with Data Envelopment Analysis , 2006 .

[13]  M. Farsi,et al.  An analysis of efficiency and productivity in Swiss hospitals , 2005 .

[14]  R. Stevenson Likelihood functions for generalized stochastic frontier estimation , 1980 .

[15]  M. S. Sebastian,et al.  Assessing the technical efficiency of health posts in rural Guatemala: a data envelopment analysis , 2014, Global health action.

[16]  Xavier Irz,et al.  DUAL TECHNOLOGICAL DEVELOPMENT IN BOTSWANA AGRICULTURE: A STOCHASTIC INPUT DISTANCE FUNCTION APPROACH / доклад на 25 конференции IAAE, Reshaping Agriculture’s Contribution to Society, International Convention Centre, Durban, South Africa, 16-23 August 2003 , 2004 .

[17]  Eyob Zere,et al.  Technical efficiency of district hospitals: Evidence from Namibia using Data Envelopment Analysis , 2006, Cost effectiveness and resource allocation : C/E.

[18]  R. Kneller,et al.  The specification of the aggregate production function in the presence of inefficiency , 2003 .

[19]  B. Yawe Hospital performance evaluation in Uganda: a super efficiency data envelope analysis model , 2010 .

[20]  Dennis L. Bricker,et al.  Profit Incentives and Technical Efficiency in the Production of Nursing Home Care , 1989 .

[21]  A. Charnes,et al.  Data Envelopment Analysis Theory, Methodology and Applications , 1995 .

[22]  Zilla Sinuany-Stern,et al.  The relationship between DEA efficiency and the type of production function, the degree of homogeneity, and error variability , 2013, Central Eur. J. Oper. Res..

[23]  Rajiv D. Banker,et al.  A simulation study of DEA and parametric frontier models in the presence of heteroscedasticity , 2004, Eur. J. Oper. Res..

[24]  Steffen Flessa,et al.  Efficiency of primary care in rural Burkina Faso. A two-stage DEA analysis , 2011, Health economics review.

[25]  William W. Cooper,et al.  Handbook on data envelopment analysis , 2011 .

[26]  T. Pham,et al.  Measuring Efficiency in Health Care: Analytic Techniques and Health Policy , 2007 .

[27]  T. Bärnighausen,et al.  Universal antiretroviral treatment: the challenge of human resources. , 2010, Bulletin of the World Health Organization.

[28]  Chunyan Yu,et al.  The effects of exogenous variables in efficiency measurement - A monte carlo study , 1998, Eur. J. Oper. Res..

[29]  Uwe Jensen,et al.  A Monte Carlo study on multiple output stochastic frontiers: a comparison of two approaches , 2014, Journal of Productivity Analysis.

[30]  G. Carrin,et al.  WHO The world health report - health systems financing: the path to universal coverage. , 2010 .

[31]  Mark A. Andor,et al.  The StoNED age: the departure into a new era of efficiency analysis? A monte carlo comparison of StoNED and the “oldies” (SFA and DEA) , 2013, Journal of Productivity Analysis.

[32]  Andrea Resti,et al.  Efficiency measurement for multi-product industries: A comparison of classic and recent techniques based on simulated data , 2000, Eur. J. Oper. Res..

[33]  R. Banker,et al.  A Monte Carlo comparison of two production frontier estimation methods: Corrected ordinary least squares and data envelopment analysis , 1993 .

[34]  Rolf Färe,et al.  Applications of Modern Production Theory: Efficiency and Productivity , 1988 .

[35]  John Ruggiero,et al.  Efficiency estimation and error decomposition in the stochastic frontier model: A Monte Carlo analysis , 1999, Eur. J. Oper. Res..

[36]  D. Mortimer Competing Methods for Efficiency Measurement : A Systematic Review of Direct DEA vs SFA/DFA Comparisons , 2002 .

[37]  A. Charnes,et al.  A Comparison of DEA and Translog Estimates of Production Frontiers Using Simulated Observations from a Known Technology , 1988 .

[38]  Sarah M. Estelle,et al.  Three-Stage DEA Models for Incorporating Exogenous Inputs , 2009, Comput. Oper. Res..

[39]  Sergio Perelman,et al.  A comparison of parametric and non-parametric distance functions: With application to European railways , 1999, Eur. J. Oper. Res..

[40]  C. Lovell,et al.  RESOURCES AND FUNCTIONINGS: A NEW VIEW OF INEQUALITY IN AUSTRALIA , 1994 .

[41]  M. S. Sebastian,et al.  Assessing the technical efficiency of health posts in rural Guatemala: a data envelopment analysis. , 2014, Global Health Action.

[42]  Timothy Coelli,et al.  On the econometric estimation of the distance function representation of a production technology , 2000 .

[43]  Stephen Jan,et al.  Task Shifting for Non-Communicable Disease Management in Low and Middle Income Countries – A Systematic Review , 2014, PloS one.

[44]  W. Greene A Gamma-distributed stochastic frontier model , 1990 .

[45]  M. Andor,et al.  A Monte Carlo simulation comparing DEA, SFA and two simple approaches to combine efficiency estimates , 2011 .

[46]  P. W. Wilson,et al.  ASYMPTOTICS AND CONSISTENT BOOTSTRAPS FOR DEA ESTIMATORS IN NONPARAMETRIC FRONTIER MODELS , 2008, Econometric Theory.

[47]  Daniel Santín,et al.  Measuring educational efficiency at student level with parametric stochastic distance functions: an application to Spanish PISA results , 2011 .

[48]  John Ruggiero,et al.  Technical efficiency estimation with multiple inputs and multiple outputs using regression analysis , 2011, Eur. J. Oper. Res..

[49]  S. Ozawa,et al.  Which Path to Universal Health Coverage? Perspectives on the World Health Report 2010 , 2010, PLoS medicine.

[50]  C. Lovell,et al.  On the estimation of technical inefficiency in the stochastic frontier production function model , 1982 .

[51]  W. Eichhorn,et al.  Models and Measurement of Welfare and Inequality , 1994 .