Incorporating preference information in a range directional composite indicator: The case of Portuguese public hospitals

Abstract Grasping the intricacy and diversity of complex systems dealing with ever-growing amounts of data is essential to public and private institutions’ continuous improvement. Composite indicators (CIs) emerge as aggregators of key performance indicators, providing a single measure that reflects those multidimensional performance aspects. One way to build such measures is based on the use of data envelopment analysis (DEA). Several DEA models can be used to generate CIs. Still, not many of them can deal concurrently with desirable and undesirable outputs, and incorporate the decision-making actors’ preference information. Based on the directional ‘Benefit-of-the-Doubt’ model, we propose a novel approach consisting of the simultaneous use of weight restrictions and an artificial target reached via a range directional vector. The resulting CI assesses the Portuguese public hospitals’ performance under two perspectives of hospital activity: users and providers. In the end, managerial and policy implications are withdrawn from the results of this study conducted in cooperation with the Portuguese Ministry of Health.

[1]  J. Pastor,et al.  Measuring macroeconomic performance in the OECD: A comparison of European and non-European countries , 1995 .

[2]  Rowena Jacobs,et al.  Measuring Efficiency in Health Care: Analytic Techniques and Health Policy , 2006 .

[3]  A. Wierzbicki.  On the completeness and constructiveness of parametric characterizations to vector optimization problems , 1986 .

[4]  Jorge Moreira da Costa,et al.  Performance Assessment of Construction Companies Integrating Key Performance Indicators and Data Envelopment Analysis , 2010 .

[5]  Giannis Karagiannis,et al.  Assessing research effectiveness: a comparison of alternative nonparametric models , 2017, J. Oper. Res. Soc..

[6]  P. Amorim,et al.  Performance benchmarking using composite indicators to support regulation of the Portuguese wastewater sector , 2020 .

[7]  Pekka Korhonen,et al.  A Careful Look at Efficiency in Multiple Objective Linear Programming , 1989 .

[8]  Lawrence M. Seiford,et al.  Modeling undesirable factors in efficiency evaluation , 2002, Eur. J. Oper. Res..

[9]  R. S. Laundy,et al.  Multiple Criteria Optimisation: Theory, Computation and Application , 1989 .

[10]  Laurens Cherchye,et al.  Constructing composite indicators with imprecise data: A proposal , 2011, Expert Syst. Appl..

[11]  Yaakov Roll,et al.  An application procedure for DEA , 1989 .

[12]  Theodor J. Stewart,et al.  DEA and MCDA: Competing or Complementary Approaches? , 1999 .

[13]  Abraham Charnes,et al.  Measuring the efficiency of decision making units , 1978 .

[14]  Ana S. Camanho,et al.  The assessment of corporate social responsibility: The construction of an industry ranking and identification of potential for improvement , 2019, Eur. J. Oper. Res..

[15]  Stefano Tarantola,et al.  Handbook on Constructing Composite Indicators: Methodology and User Guide , 2005 .

[16]  Barton A. Smith,et al.  Comparative Site Evaluations for Locating a High-Energy Physics Lab in Texas , 1986 .

[17]  A. Donabedian.  Evaluating the quality of medical care. 1966. , 1966, The Milbank quarterly.

[18]  Rolf Färe,et al.  Benefit-of-the-doubt aggregation and the diet problem , 2014 .

[19]  Joseph C. Paradi,et al.  Establishing the "practical frontier" in data envelopment analysis , 2004 .

[20]  V. V. Podinovski.  Improving data envelopment analysis by the use of production trade-offs , 2007, J. Oper. Res. Soc..

[21]  B. Golany,et al.  Controlling Factor Weights in Data Envelopment Analysis , 1991 .

[22]  Abraham Charnes,et al.  Cone ratio data envelopment analysis and multi-objective programming , 1989 .

[23]  R. Färe.  Derivation of Shadow Prices for Undesirable Outputs: A Distance Function Approach , 1993 .

[24]  Cláudia S. Sarrico,et al.  Restricting virtual weights in data envelopment analysis , 2004, Eur. J. Oper. Res..

[25]  R. Dyson,et al.  Reducing Weight Flexibility in Data Envelopment Analysis , 1988 .

[26]  José Rui Figueira,et al.  On the Choquet multiple criteria preference aggregation model: Theoretical and practical insights from a real-world application , 2018, Eur. J. Oper. Res..

[27]  Rui Cunha Marques,et al.  Do quality and access to hospital services impact on their technical efficiency? , 2019, Omega.

[28]  P. Loirat,et al.  Constructing a Composite Quality Score for the Care of Acute Myocardial Infarction Patients at Discharge: Impact on Hospital Ranking , 2011, Medical care.

[29]  Rolf Färe,et al.  A benefit-of-the-doubt model with reverse indicators , 2019, Eur. J. Oper. Res..

[30]  Wade D. Cook,et al.  Evaluating power plant efficiency: a hierarchical model , 2005, Comput. Oper. Res..

[31]  Emmanuel Thanassoulis,et al.  Simulating Weights Restrictions in Data Envelopment Analysis by Means of Unobserved Dmus , 1998 .

[32]  Tom Van Puyenbroeck,et al.  On the Output Orientation of the Benefit-of-the-Doubt-Model , 2018 .

[33]  Rui Cunha Marques,et al.  Using a Choquet integral-based approach for incorporating decision-maker's preference judgments in a Data Envelopment Analysis model , 2020, Eur. J. Oper. Res..

[34]  R. Shepherd.  Theory of cost and production functions , 1970 .

[35]  Nicky Rogge,et al.  Waste Performance of NUTS 2-regions in the EU: A Conditional Directional Distance Benefit-of-the-Doubt Model , 2017 .

[36]  Emmanuel Thanassoulis,et al.  Weights restrictions and value judgements in Data Envelopment Analysis: Evolution, development and future directions , 1997, Ann. Oper. Res..

[37]  R. Marques,et al.  A critical look at the Portuguese public–private partnerships in healthcare , 2020, The International journal of health planning and management.

[38]  A. Camanho,et al.  A temporal progressive analysis of the social performance of mining firms based on a Malmquist index estimated with a Benefit-of-the-Doubt directional model , 2020 .

[39]  Giorgia Oggioni,et al.  Eco-efficiency of the world cement industry: A data envelopment analysis , 2011 .

[40]  Yasar A. Ozcan,et al.  Health Care Benchmarking and Performance Evaluation: An Assessment Using Data Envelopment Analysis (DEA) , 2007 .

[41]  Ana S. Camanho,et al.  Undesirable outputs and weighting schemes in composite indicators based on data envelopment analysis , 2015, Eur. J. Oper. Res..

[42]  Edward F. McQuarrie,et al.  Focus Groups: Theory and Practice , 1991 .

[43]  Maria C. Andrade e Silva,et al.  Benchmarking of secondary schools based on Students’ results in higher education , 2020, Omega.

[44]  Pekka J. Korhonen,et al.  Restricting weights in value efficiency analysis , 2000, Eur. J. Oper. Res..

[45]  Joe Zhu.  Data Envelopment Analysis with Preference Structure , 1996 .

[46]  S. De Jaeger,et al.  Identifying the most relevant peers for benchmarking waste management performance: A conditional directional distance Benefit-of-the-Doubt approach. , 2019, Waste management.

[47]  Victor V. Podinovski,et al.  DEA models for the explicit maximisation of relative efficiency , 2001, Eur. J. Oper. Res..

[48]  D. Morgan.  Focus groups for qualitative research. , 1988, Hospital guest relations report.

[49]  Pekka Korhonen,et al.  Extension of Data Envelopment Analysis with Preference Information: Value Efficiency , 2015 .

[50]  B. Hollingsworth.  Non-Parametric and Parametric Applications Measuring Efficiency in Health Care , 2003, Health care management science.

[51]  Nicky Rogge,et al.  Quality of Life in the European Union: A Multidimensional Analysis , 2019 .

[52]  Emmanuel Thanassoulis,et al.  Estimating preferred target input−output levels using data envelopment analysis , 1992 .

[53]  Boaz Golany,et al.  An Interactive MOLP Procedure for the Extension of DEA to Effectiveness Analysis , 1988 .

[54]  J. Wallenius,et al.  A Value Efficiency Approach to Incorporating Preference Information in Data Envelopment Analysis , 1999 .

[55]  Y. Ozcan.  Health Care Benchmarking and Performance Evaluation , 2008 .

[56]  John E. Beasley,et al.  Restricting Weight Flexibility in Data Envelopment Analysis , 1990 .

[57]  Richard A. Krueger,et al.  When to Use Focus Groups and Why , 1993 .

[58]  C. Lovell,et al.  Measuring the macroeconomic performance of the Taiwanese economy , 1995 .

[59]  Russell Pittman,et al.  Multilateral Productivity Comparisons with Undesirable Outputs , 1983 .

[60]  Jyrki Wallenius,et al.  Can a linear value function explain choices? An experimental study , 2012, Eur. J. Oper. Res..

[61]  Cláudia S. Sarrico,et al.  Pitfalls and protocols in DEA , 2001, Eur. J. Oper. Res..

[62]  Laurens Cherchye,et al.  An Introduction to ‘Benefit of the Doubt’ Composite Indicators , 2007 .

[63]  J. Church.  Human Development Report , 2001 .

[64]  Holger Scheel,et al.  Undesirable outputs in efficiency valuations , 2001, Eur. J. Oper. Res..

[65]  Valentina Ferretti,et al.  From stakeholders analysis to cognitive mapping and Multi-Attribute Value Theory: An integrated approach for policy support , 2016, Eur. J. Oper. Res..

[66]  Rui Cunha Marques,et al.  Using multi-criteria decision analysis to rank European health systems: The Beveridgian financing case , 2020 .

[67]  B. Golany.  A note on including ordinal relations among multipliers in data envelopment analysis , 1988 .

[68]  Ralph L. Keeney,et al.  Value-Focused Thinking: A Path to Creative Decisionmaking , 1992 .

[69]  Laura Carosi,et al.  Global Public spending efficiency in Tuscan municipalities , 2018 .

[70]  Rolf Färe,et al.  Productivity and Undesirable Outputs: A Directional Distance Function Approach , 1995 .

[71]  Russell G. Thompson,et al.  The role of multiplier bounds in efficiency analysis with application to Kansas farming , 1990 .

[72]  Luis C. Dias,et al.  An application of value-based DEA to identify the best practices in primary health care , 2016, OR Spectr..

[73]  C. Mateus,et al.  Cost effects of hospital mergers in Portugal , 2014, The European Journal of Health Economics.

[74]  L. R. Christensen,et al.  MULTILATERAL COMPARISONS OF OUTPUT, INPUT, AND PRODUCTIVITY USING SUPERLATIVE INDEX NUMBERS* , 1982 .

[75]  L. Seiford,et al.  Strict vs. weak ordinal relations for multipliers in data envelopment analysis , 1991 .

[76]  Jean-Luc Marichal,et al.  Determination of weights of interacting criteria from a reference set , 2000, Eur. J. Oper. Res..