Creating composite indicators with DEA and robustness analysis: the case of the Technology Achievement Index

Composite indicators (CIs) are often used for benchmarking countries' performance, but they frequently stir controversies about the unavoidable subjectivity in their construction. Data Envelopment Analysis helps to overcome some key limitations, as it does not need any prior information on either the normalization of sub-indicators or on an agreed unique set of weights. Still, subjective decisions remain, and such modelling uncertainty propagates onto countries' CI scores and rankings. Uncertainty and sensitivity analysis are therefore needed to assess the robustness of the final outcome and to analyse how much each source of uncertainty contributes to the output variance. The current paper reports on these issues, using the Technology Achievement Index as an illustration.

[1]  Mark McGillivray,et al.  Understanding human well-being , 2007 .

[2]  C. Kao,et al.  Data envelopment analysis with common weights: the compromise solution approach , 2005, J. Oper. Res. Soc..

[3]  Emmanuel Thanassoulis,et al.  Incorporating Value Judgments in DEA , 2004 .

[4]  Giuseppe Munda,et al.  Version November , 2003 On the Methodological Foundations of Composite Indicators Used for Ranking Countries , 2003 .

[5]  Kaoru Tone,et al.  Data Envelopment Analysis , 1996 .

[6]  A. Saltelli,et al.  Making best use of model evaluations to compute sensitivity indices , 2002 .

[7]  Dimitris K. Despotis,et al.  A reassessment of the human development index via data envelopment analysis , 2005, J. Oper. Res. Soc..

[8]  Stefano Tarantola,et al.  Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators , 2005 .

[9]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .

[10]  Michael Obersteiner,et al.  Remeasuring the HDI by Data Envelopement Analysis , 2001 .

[11]  Timo Kuosmanen,et al.  Benchmarking Sustainable Development : A Synthetic Meta-index Approach , 2002 .

[12]  C. Lovell,et al.  One Market, One Number? A Composite Indicator Assessment of EU Internal Market Dynamics , 2005 .

[13]  Wade D. Cook,et al.  A multiple-criteria composite index model for quantitative and qualitative data , 1994 .

[14]  J. Florens,et al.  Nonparametric frontier estimation: a robust approach , 2002 .

[15]  Emmanuel Thanassoulis,et al.  Weights restrictions and value judgements in Data Envelopment Analysis: Evolution, development and future directions , 1997, Ann. Oper. Res..

[16]  David J. Groggel,et al.  Practical Nonparametric Statistics , 2000, Technometrics.

[17]  Michael Freudenberg,et al.  Composite Indicators of Country Performance: A Critical Assessment , 2003 .

[18]  E. Sheldon,et al.  Social Indicators , 1975, Science.

[19]  Stefano Tarantola,et al.  Handbook on Constructing Composite Indicators: Methodology and User Guide , 2005 .

[20]  Léopold Simar,et al.  Detecting Outliers in Frontier Models: A Simple Approach , 2003 .

[21]  Elsie M. Sunderland,et al.  Draft Guidance on the Development, Evaluation, and Application of Regulatory Environmental Models , 2003 .

[22]  Joseph A. C. Delaney.  Sensitivity analysis , 2018, The African Continental Free Trade Area: Economic and Distributional Effects.

[23]  Sakiko Fukuda-Parr,et al.  Measuring the Technology Achievement of Nations and the Capacity to Participate in the Network Age , 2002 .

[24]  Cláudia S. Sarrico,et al.  Restricting virtual weights in data envelopment analysis , 2004, Eur. J. Oper. Res..

[25]  John E. Beasley,et al.  Restricting Weight Flexibility in Data Envelopment Analysis , 1990 .

[26]  Abraham Charnes,et al.  Preface to topics in data envelopment analysis , 1984, Ann. Oper. Res..

[27]  A. Sen.  On Economic Inequality , 1974 .

[28]  J. Pastor,et al.  Measuring macroeconomic performance in the OECD: A comparison of European and non-European countries , 1995 .

[29]  A. Atkinson.  Social Indicators: The EU and Social Inclusion , 2002 .

[30]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice , 2002 .

[31]  A. U.S.,et al.  Measuring the efficiency of decision making units , 2003 .

[32]  Joe Zhu.  Data Envelopment Analysis with Preference Structure , 1996 .

[33]  I. Sobol.  On the distribution of points in a cube and the approximate evaluation of integrals , 1967 .

[34]  Rajiv D. Banker,et al.  Chapter 11 Simulation studies of efficiency, returns to scale and misspecification with nonlinear functions in DEA , 1996, Ann. Oper. Res..

[35]  R. W. Saaty,et al.  The analytic hierarchy process—what it is and how it is used , 1987 .

[36]  Hans Bjurek,et al.  Benchmarking European labour market performance with efficiency frontier techniques , 2000 .

[37]  Sidney Addelman,et al.  trans-Dimethanolbis(1,1,1-trifluoro-5,5-dimethylhexane-2,4-dionato)zinc(II) , 2008, Acta crystallographica. Section E, Structure reports online.

[38]  P. W. Wilson,et al.  Sensitivity Analysis of Efficiency Scores: How to Bootstrap in Nonparametric Frontier Models , 1998 .

[39]  Vivian Valdmanis,et al.  Sensitivity analysis for DEA models: An empirical example using public vs. NFP hospitals , 1992 .

[40]  L. Cherchye,et al.  Legitimately Diverse, Yet Comparable: On Synthesizing Social Inclusion Performance in the EU , 2004 .

[41]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models , 2004 .

[42]  Paul W. Wilson,et al.  Detecting influential observations in data envelopment analysis , 1995 .

[43]  Francisco Pedraja-Chaparro,et al.  On the Role of Weight Restrictions in Data Envelopment Analysis , 1997 .

[44]  Stefano Tarantola,et al.  Tools for Composite Indicators Building , 2005 .

[45]  Tarantola Stefano,et al.  European Innovation Scoreboard 2006 - Comparative Analysis of Innovation Performance , 2007 .

[46]  Renato De Leone,et al.  Data Envelopment Analysis , 2009, Encyclopedia of Optimization.