Robust measurement of innovation performances in Europe with a hierarchy of interacting composite indicators

For long time the measurement of innovation has been in the forefront of policy makers' and researchers' agenda worldwide. Therefore, there is an ongoing debate about which indicators should be used to measure innovation. Recent approaches have favoured the use of composite innovation indicators. However, there is no consensus about the appropriate methodology to aggregate the varying dimensions of innovation into a single summary indicator. One of the best known examples of composite innovation indicators is the European Innovation Scoreboard (EIS). It is a relevant tool for benchmarking innovation in Europe. Still, the EIS lacks a proper scheme for weighting the included indicators according to their relative importance. In this context, we propose an appraisal methodology permitting to take into consideration the interaction of criteria and robustness concerns related to the elicitation of the weights assigned to the elementary indicators. With this aim, we apply the hierarchical-SMAA-Choquet integral approach. This integrated multicriteria decision making (MCDM) method helps the users to rank and benchmark countries' innovation performance taking into account the importance and interaction of criteria assigned by themselves, rather than equal weights or weights exogenously fixed by external experts.

[1]  Abraham Charnes,et al.  Measuring the efficiency of decision making units , 1978 .

[2]  Matthias Ehrgott,et al.  Multiple criteria decision analysis: state of the art surveys , 2005 .

[3]  H. Abdi,et al.  Principal component analysis , 2010 .

[4]  G. Rota.  On the Foundations of Combinatorial Theory , 2009 .

[5]  Marijana Petrovic,et al.  An ELECTRE-based decision aid tool for stepwise benchmarking: An application over EU Digital Agenda targets , 2014, Decis. Support Syst..

[6]  T. Dantsis,et al.  A methodological approach to assess and compare the sustainability level of agricultural plant production systems , 2010 .

[7]  Gottfried Tappeiner,et al.  Measuring regional innovation: A critical inspection of the ability of single indicators to shape technological change , 2017 .

[8]  M. Grabisch.  The application of fuzzy integrals in multicriteria decision making , 1996 .

[9]  Laurens Cherchye,et al.  Creating composite indicators with DEA and robustness analysis: the case of the Technology Achievement Index , 2006, J. Oper. Res. Soc..

[10]  T. Holgersson,et al.  Towards a multivariate innovation index , 2018 .

[11]  Salvatore Greco,et al.  An Overview of ELECTRE Methods and their Recent Extensions , 2013 .

[12]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  Henry Etzkowitz,et al.  The Triple Helix as a model for innovation studies , 1998 .

[14]  Zvi Griliches,et al.  R&D, Patents, and Productivity , 1981 .

[15]  Ching-Lai Hwang,et al.  Methods for Multiple Attribute Decision Making , 1981 .

[16]  S. Greco,et al.  On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness , 2019 .

[17]  G. Rota.  On the foundations of combinatorial theory I. Theory of Möbius Functions , 1964 .

[18]  P. Nijkamp,et al.  The attractiveness and competitiveness of tourist destinations: A study of Southern Italian regions , 2009 .

[19]  S. Greco,et al.  Robust Ordinal Regression and Stochastic Multiobjective Acceptability Analysis in multiple criteria hierarchy process for the Choquet integral preference model , 2016 .

[20]  Frane Adam,et al.  Measuring National Innovation Performance: The Innovation Union Scoreboard Revisited , 2013 .

[21]  B. Lanvin,et al.  The Global Innovation Index 2011 , 2011 .

[22]  G. Choquet.  Theory of capacities , 1954 .

[23]  Carmelo Maria Torre,et al.  Non-compensatory composite indicators for the evaluation of urban planning policy: The Land-Use Policy Efficiency Index (LUPEI) , 2018, Eur. J. Oper. Res..

[24]  Salvatore Greco,et al.  Rough sets theory for multicriteria decision analysis , 2001, Eur. J. Oper. Res..

[25]  C. Edquist,et al.  The Innovation Union Scoreboard is Flawed: The case of Sweden – not being the innovation leader of the EU , 2015 .

[26]  Yorgos Goletsis,et al.  A multilevel and multistage efficiency evaluation of innovation systems: A multiobjective DEA approach , 2016, Expert Syst. Appl..

[27]  M. Grabisch,et al.  A representation of preferences by the Choquet integral with respect to a 2-additive capacity , 2011 .

[28]  Milosz Kadzinski,et al.  Scoring procedures for multiple criteria decision aiding with robust and stochastic ordinal regression , 2016, Comput. Oper. Res..

[29]  T. L. Saaty.  A Scaling Method for Priorities in Hierarchical Structures , 1977 .

[30]  S. Greco,et al.  Stochastic multi-attribute acceptability analysis (SMAA): an application to the ranking of Italian regions , 2018 .

[31]  G. Mauris,et al.  Quantitative expression and aggregation of performance measurements based on the MACBETH multi-criteria method , 2007 .

[32]  H. Grupp,et al.  Indicators for national science and technology policy: how robust are composite indicators? , 2004 .

[33]  H. Grupp,et al.  Review and new evidence on composite innovation indicators for evaluating national performance , 2010 .

[34]  Salvatore Greco,et al.  Evaluating and comparing entrepreneurial ecosystems using SMAA and SMAA-S , 2018, The Journal of Technology Transfer.

[35]  Heinz Hollenstein,et al.  A composite indicator of a firm's innovativeness. An empirical analysis based on survey data for Swiss manufacturing , 1996 .

[36]  S. Tarantola,et al.  State-of-the-art Report on Current Methodologies and Practices for Composite Indicator Development , 2002 .

[37]  Yorgos Goletsis,et al.  Composite innovation metrics: MCDA and the Quadruple Innovation Helix framework , 2017, Technological Forecasting and Social Change.

[38]  Stefano Tarantola,et al.  Methodology report on European Innovation Scoreboard , 2005 .

[39]  Salvatore Greco,et al.  Robust sustainable development assessment with composite indices aggregating interacting dimensions: The hierarchical-SMAA-Choquet integral approach , 2018, Knowl. Based Syst..

[40]  Salvatore Greco,et al.  Multiple Criteria Hierarchy Process in Robust Ordinal Regression , 2012, Decis. Support Syst..

[41]  H. Hollanders,et al.  An assessment of the Innovation Union Scoreboard as a tool to analyse national innovation capacities: The case of Switzerland , 2015 .

[42]  Jan Kozlowski,et al.  Innovation indices: the need for positioning them where they properly belong , 2015, Scientometrics.

[43]  A. Schibany,et al.  The European Innovation Scoreboard: drowning by numbers? , 2008 .

[44]  B F Giannetti,et al.  The reliability of experts' opinions in constructing a composite environmental index: the case of ESI 2005. , 2009, Journal of environmental management.

[45]  Andreas Pyka,et al.  Ranking the performance of national innovation systems in the Iberian Peninsula and Latin America from a neo-Schumpeterian economics perspective , 2014 .

[46]  Nardo Michela,et al.  Constructing Consistent Composite Indicators: the Issue of Weights , 2005 .

[47]  P. Vincke,et al.  Note-A Preference Ranking Organisation Method: The PROMETHEE Method for Multiple Criteria Decision-Making , 1985 .

[48]  S French,et al.  Multicriteria Methodology for Decision Aiding , 1996 .

[49]  J. Figueira,et al.  A survey on stochastic multicriteria acceptability analysis methods , 2008 .

[50]  Ralph L. Keeney,et al.  Decisions with multiple objectives: preferences and value tradeoffs , 1976 .

[51]  Risto Lahdelma,et al.  SMAA - Stochastic multiobjective acceptability analysis , 1998, Eur. J. Oper. Res..

[52]  Tan Yigitcanlar,et al.  Towards a sustainable industrial ecology: Implementation of a novel approach in the performance evaluation of Italian regions , 2018 .

[53]  Andrea Saltelli,et al.  Ratings and rankings: voodoo or science? , 2011, 1104.3009.

[54]  L. S. Shapley,et al.  17. A Value for n-Person Games , 1953 .

[55]  C. B. E. Costa,et al.  MACBETH — An Interactive Path Towards the Construction of Cardinal Value Functions , 1994 .

[56]  Viktor Pocajt,et al.  A differential multi-criteria analysis for the assessment of sustainability performance of European countries: Beyond country ranking , 2017 .

[57]  Luis C. Dias,et al.  Resolving inconsistencies among constraints on the parameters of an MCDA model , 2003, Eur. J. Oper. Res..

[58]  L. Leydesdorff,et al.  The dynamics of innovation: from National Systems and , 2000 .

[59]  Trinidad Gómez,et al.  Building composite indicators using multicriteria methods: a review , 2019 .