Meta-choices in ranking knowledge-based organizations

PurposeThe purpose of this paper is to address the issue of knowledge visualization and its connection with performance measurement from an epistemological point of view, considering quantification and measurement not just as technical questions but showing their relevant implications on the management decision-making of knowledge-based organizations.Design/methodology/approachThis study proposes a theoretical contribution that combines two lines of research for identifying the three main meta-choices problems that arise in the multidimensional benchmarking of knowledge-based organizations. The first is the meta-choice problem related to the choice of the algorithm used (Iazzolino et al., 2012; Laise et al., 2015; Daraio, 2017a). The second refers to the choice of the variables to be included in the model (Daraio, 2017a). The third concerns the choice of the data on which the analyses are carried out (Daraio, 2017a).FindingsThe authors show the interplay existing among the three meta-choices in multidimensional benchmarking, considering as key performance indicators intellectual capital, including Human Capital, Structural Capital and Relational Capital, and performances, evaluated in financial and non-financial terms. This study provides an empirical analysis on Italian Universities, comparing the ranking distributions obtained by several efficiency and multi-criteria methods.Originality/valueThis study demonstrates the difficulties of the “implementation problem” in performance measurement, related to the subjectivity of results of the evaluation process when there are many evaluation criteria, and proposes the adoption of the technologies of humility related to the awareness that we can only achieve “satisficing” results.

[1]  Majid Ramezan,et al.  Intellectual capital and organizational organic structure in knowledge society: How are these concepts related? , 2011, Int. J. Inf. Manag..

[2]  C. Tan,et al.  University rankings: A review of methodological flaws , 2020 .

[3]  Shaher Momani,et al.  Are university rankings useful to improve research? A systematic review , 2018, PloS one.

[4]  Laura Marraro,et al.  Metachoice for benchmarking: a case study , 2015 .

[5]  Á. Tejada,et al.  Recognition of intellectual capital importance in the university sector , 2013 .

[6]  David Murray,et al.  Business Decision Making , 1983 .

[7]  Jian-Bo Yang,et al.  Multiple Attribute Decision Making , 1998 .

[8]  Wolfgang Kienreich,et al.  What is Knowledge Visualization? Perspectives on an Emerging Discipline , 2011, 2011 15th International Conference on Information Visualisation.

[9]  C. Corsi,et al.  A quality evaluation approach to disclosing third mission activities and intellectual capital in Italian universities , 2018 .

[10]  Matthias Ehrgott,et al.  Multiple criteria decision analysis: state of the art surveys , 2005 .

[11]  P. A. Losty,et al.  A Behavioural Theory of the Firm , 1965 .

[12]  Ángel Tejada Ponce,et al.  Intellectual capital in Spanish public universities: stakeholders' information needs , 2011 .

[13]  Léopold Simar,et al.  Central Limit Theorems for Conditional Efficiency Measures and Tests of the ‘Separability’ Condition in Non�?Parametric, Two�?Stage Models of Production , 2018 .

[14]  María Rodríguez García,et al.  Intellectual capital, organisational performance and competitive advantage , 2020, European J. of International Management.

[15]  Léopold Simar,et al.  Advanced Robust and Nonparametric Methods in Efficiency Analysis: Methodology and Applications , 2007 .

[16]  M. Sánchez,et al.  Intellectual capital dynamics in universities: a reporting model , 2009 .

[17]  Cinzia Daraio,et al.  Econometric Approaches to the Measurement of Research Productivity , 2019, Springer Handbook of Science and Technology Indicators.

[18]  A. Charnes,et al.  Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis , 1984 .

[19]  Karl-Heinz Leitner,et al.  Intellectual capital reporting for universities: conceptual background and application for Austrian universities , 2004 .

[20]  H. Simon,et al.  Rationality as Process and as Product of Thought , 1978 .

[21]  C. Daraio.  A Framework for the Assessment and Consolidation of Productivity Stylized Facts , 2020 .

[22]  H. Simon,et al.  A Behavioral Model of Rational Choice , 1955 .

[23]  James G. March,et al.  Organizational decision making: Understanding how decisions happen in organizations , 1996 .

[24]  Maneck S. Wadia.  The nature and scope of management , 1966 .

[25]  John Dumay.  A Critical Reflection on the Future of Intellectual Capital: From Reporting to Disclosure , 2016 .

[26]  María Teresa Lamata,et al.  On rank reversal and TOPSIS method , 2012, Math. Comput. Model..

[27]  E. Zavadskas,et al.  Optimization of Weighted Aggregated Sum Product Assessment , 2012 .

[28]  Fernando Martín‐Alcázar,et al.  Conceptualizing academic intellectual capital: definition and proposal of a measurement scale , 2019, Journal of Intellectual Capital.

[29]  G. Elia,et al.  Intangible assets in higher education and research: mission, performance or both? , 2010 .

[30]  Léopold Simar,et al.  Rankings and university performance: A conditional multidimensional approach , 2015, Eur. J. Oper. Res..

[31]  Po-Young Chu,et al.  Intellectual capital: An empirical study of ITRI , 2006 .

[32]  K. Knorr-Cetina,et al.  Epistemic cultures : how the sciences make knowledge , 1999 .

[33]  Cinzia Daraio.  Assessing Research and its Impacts: The Generalized Implementation Problem and a Doubly-Conditional Performance Evaluation Model , 2017, ISSI.

[34]  Gökçen Arkalı Olcay,et al.  Technological Forecasting & Social Change Is measuring the knowledge creation of universities possible ? : A review of university rankings , 2016 .

[35]  Martin J. Eppler.  What is an Effective Knowledge Visualization? Insights from a Review of Seminal Concepts , 2011, 2011 15th International Conference on Information Visualisation.

[36]  H. Hsu,et al.  Intellectual Capital , 2010 .

[37]  Susana Elena Pérez,et al.  An Intellectual Capital framework to measure universities' third mission activities , 2017 .

[38]  John Carson,et al.  Quantification – Affordances and Limits , 2020, Scholarly Assessment Reports.

[39]  Sheila Jasanoff,et al.  Technologies of humility , 2007, Nature.

[40]  E. Zavadskas,et al.  Project management by multimoora as an instrument for transition economies , 2010 .

[41]  William H.A. Johnson,et al.  An integrative taxonomy of intellectual capital: measuring the stock and flow of intellectual capital components in the firm , 1999 .

[42]  A M Weinberg,et al.  The age of substitutability. , 1976, Science.

[43]  Shlomo Zilberstein,et al.  Models of Bounded Rationality , 1995 .

[44]  Yolanda Ramírez Córcoles.  Importance of intellectual capital disclosure in Spanish universities , 2013 .

[45]  L. Edvinsson,et al.  Developing a model for managing intellectual capital , 1996 .

[46]  R. Anthony.  The trouble with profit maximization , 2007 .

[47]  Abraham Charnes,et al.  Measuring the efficiency of decision making units , 1978 .

[48]  Ivoni Bezhani.  Intellectual capital reporting at UK universities , 2010 .

[49]  Léopold Simar,et al.  Testing Hypotheses in Nonparametric Models of Production , 2016 .

[50]  Domenico Laise,et al.  Business multicriteria performance analysis: a tutorial , 2012 .

[51]  R. Kaplan,et al.  Using the balanced scorecard as a strategic management system , 1996 .

[52]  Cinzia Daraio,et al.  A Framework for the Assessment of Research and Its Impacts , 2017, J. Data Inf. Sci..