The differing meanings of indicators under different policy contexts. The case of internationalisation

In this chapter we build upon Moed’s conceptual contributions on the importance of the policy context when using and interpreting scientometric indicators. We focus on the use of indicators in research evaluation regarding internationalisation policies. The globalisation of higher education presents important challenges to institutions worldwide, which are confronted with tensions derived from the need to respond both to their local necessities and demands while participating in global networks. In this context, indicators have served as measures for monitoring and enforcing internationalisation policies, in many cases interpreting them regardless of the policy context in which they are enforced. We will analyse three examples of indicators related to internationalisation. The first one is about international collaborations, under the assumption that a greater number of internationally co-authored publications will benefit a national science system as it will result in higher citation impact. The second one relates to the promotion of English language as the dominant language of science. The third case analyses how different policy contexts shape the selection and construction of indicators, sometimes in a partial way which does not properly reflect the phenomenon under study. The examples illustrate that the interpretation and policy implications of the ‘same’ S&T indicators differ depending on specific contexts.

[1]  Gunnar Sivertsen,et al.  Patterns of internationalization and criteria for research assessment in the social sciences and humanities , 2016, Scientometrics.

[2]  Jean-Baptiste Meyer,et al.  Network Approach versus Brain Drain: Lessons from the Diaspora , 2001 .

[3]  Maresi Nerad,et al.  Globalization and the Internationalization of Graduate Education: A Macro and Micro View , 2010 .

[4]  G. Bassioni,et al.  Entice Africa's scientists to stay , 2016, Nature.

[5]  Thed N. van Leeuwen,et al.  New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications , 1995, Scientometrics.

[6]  Anton J. Nederhof,et al.  Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review , 2006, Scientometrics.

[7]  Wolfgang Glänzel,et al.  Inflationary bibliometric values: The role of scientific collaboration and the need for relative indicators in evaluative studies , 2004, Scientometrics.

[8]  Vincent Larivière,et al.  The many faces of mobility: Using bibliometric data to measure the movement of scientists , 2018, J. Informetrics.

[9]  Giovanni Abramo,et al.  Do interdisciplinary research teams deliver higher gains to science? , 2017, Scientometrics.

[10]  Domingo Docampo,et al.  Mining university rankings: Publication output and citation impact as their basis , 2019, Research Evaluation.

[11]  V. Larivière,et al.  Le français, langue seconde ? De l'évolution des lieux et langues de publication des chercheurs au Québec, en France et en Allemagne , 2019, Recherche.

[12]  Jonathan Adams Collaborations: The rise of research networks , 2012, Nature.

[13]  Jean-Claude Guédon,et al.  Excellence or quality? Impact of the current competition regime on science and scientific publishing in Latin America and its implications for development , 2014 .

[14]  Cassidy R. Sugimoto,et al.  Towards a global scientific brain: Indicators of researcher mobility using co-affiliation data , 2016, ArXiv.

[15]  Bryan J. Robinson,et al.  Flying in the face of illusion. A comparative study of the variables that interact in English-language scientific journals publishing translations. , 2012 .

[16]  Henk F. Moed,et al.  Applied Evaluative Informetrics , 2017, Qualitative and Quantitative Analysis of Scientific and Scholarly Communication.

[17]  H. Moed,et al.  The use of bibliometric data for the measurement of university research performance , 1985 .

[18]  Robert J. W. Tijssen,et al.  In search of the European Paradox: an international comparison of Europe's scientific performance and knowledge flows in information and communication technologies research , 1999 .

[19]  Alexander M. Petersen,et al.  Quantifying the negative impact of brain drain on the integration of European science , 2017, Science Advances.

[20]  Thed N. van Leeuwen,et al.  A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact , 2010, J. Assoc. Inf. Sci. Technol..

[21]  Rafael Aleixandre-Benavent,et al.  The Impact Factor in non-English-speaking countries , 2012, Scientometrics.

[22]  Loet Leydesdorff,et al.  Network Structure, Self-Organization and the Growth of International Collaboration in Science.Research Policy, 34(10), 2005, 1608-1618. , 2005, 0911.4299.

[23]  Valeria Aman A new bibliometric approach to measure knowledge transfer of internationally mobile scientists , 2018, Scientometrics.

[24]  A. Raan Science as an international enterprise , 1997 .

[25]  E. Hazelkorn Rankings and the Reshaping of Higher Education: The Battle for World-Class Excellence , 2011 .

[26]  Robert J. W. Tijssen,et al.  Chinese researchers returning home: Impacts of international mobility on research collaboration and scientific productivity , 2008, Scientometrics.

[27]  Nicolás Robinson-García,et al.  Global research collaboration: Networks and partners in South East Asia , 2017, ArXiv.

[28]  L. Leydesdorff,et al.  The Continuing Growth of Global Cooperation Networks in Research: A Conundrum for National Governments , 2015, PloS one.

[29]  Louise Ackers,et al.  Moving People and Knowledge: Scientific Mobility in the European Union1 , 2005 .

[30]  Steven W. Floyd,et al.  Are managers from Mars and academicians from venus? Toward an understanding of the relationship between academic quality and practical relevance , 2004 .

[31]  Nicholas A. Bowman,et al.  U.S. News & World Report College Rankings: Modeling Institutional Effects on Organizational Reputation , 2009, American Journal of Education.

[32]  Brian Sloan,et al.  European research policy and bibliometric indicators, 1990–2005 , 2010, Scientometrics.

[33]  Henk F. Moed,et al.  A bibliometric approach to tracking international scientific migration , 2014, Scientometrics.

[34]  Thed N. van Leeuwen,et al.  Using Altmetrics for Contextualised Mapping of Societal Impact: From Hits to Networks , 2016 .

[35]  Grit Laudel,et al.  Studying the brain drain: Can bibliometric methods help? , 2003, Scientometrics.

[36]  Henk F. Moed,et al.  A critical comparative analysis of five world university rankings , 2016, Scientometrics.

[37]  Paula E. Stephan,et al.  Are the Foreign Born a Source of Strength for U.S. Science? , 1999, Science.

[38]  Adrián A. Díaz-Faes,et al.  Towards a second generation of ‘social media metrics’: Characterizing Twitter communities of attention around science , 2019, PloS one.

[39]  Loet Leydesdorff,et al.  International collaboration in science and the formation of a core group , 2008, J. Informetrics.

[40]  Kathrin Zippel,et al.  Women in Global Science: Advancing Academic Careers through International Collaboration , 2017 .

[41]  Zeng Lin,et al.  Imported talents: demographic characteristics, achievement and job satisfaction of foreign born full time faculty in four-year American colleges , 2009 .

[42]  Henk F. Moed,et al.  Does Research Mobility Have an Effect on Productivity and Impact , 2016 .

[43]  Ismael Rafols,et al.  Why Researchers Publish in Non-Mainstream Journals: Training, Knowledge Bridging and Gap-Filling , 2017 .

[44]  Vicente P. Guerrero-Bote,et al.  Quantifying the benefits of international scientific collaboration , 2013, J. Assoc. Inf. Sci. Technol..

[45]  Pablo Kreimer,et al.  Scientific co-operation and centre-periphery relations: attitudes and interests of European and Latin American scientists , 2019, Tapuya: Latin American Science, Technology and Society.

[46]  C. Wagner,et al.  Open countries have strong science , 2017, Nature.

[47]  Ismael Rafols,et al.  How journal rankings can suppress interdisciplinary research: A comparison between Innovation Stud , 2012 .

[48]  D. Libaers Foreign-Born Academic Scientists and Their Interactions with Industry: Implications for University Technology Commercialization and Corporate Innovation Management , 2014 .

[49]  Javier Otamendi,et al.  Measuring and assessing researcher mobility from CV analysis: the case of the Ramón y Cajal programme in Spain , 2008 .

[50]  D. Hicks,et al.  Reception of Spanish sociology by domestic and foreign audiences differs and has consequences for evaluation , 2015 .

[51]  Paula E. Stephan,et al.  Exceptional contributions to US science by the foreign-born and foreign-educated , 2001 .

[52]  Paula E. Stephan,et al.  Context Factors and the Performance of Mobile Individuals in Research Teams , 2017 .

[53]  Rodrigo Costas,et al.  Studying the Scientific Mobility and International Collaboration Funded by the China Scholarship Council , 2022, ISSI.

[54]  Brigitte Tiefenthaler,et al.  Counting quality? The Czech performance-based research funding system , 2015 .

[55]  Henk F. Moed,et al.  Researchers’ Mobility, Productivity and Impact: Case of Top Producing Authors in Seven Disciplines , 2016 .

[56]  Thed N. van Leeuwen,et al.  Language biases in the coverage of the Science Citation Index and its consequencesfor international comparisons of national research performance , 2001, Scientometrics.

[57]  N. Robinson-García,et al.  On the Dominance of Quantitative Evaluation in ‘Peripheral’ Countries: Auditing Research with Technologies of Distance , 2016 .

[58]  Gunnar Sivertsen,et al.  Multilingüisme equilibrat en ciència , 2018 .

[59]  Li Tang,et al.  The penalty of containing more non-English articles , 2017, Scientometrics.

[60]  E. Welch,et al.  Comparing the collaboration networks and productivity of China-born and US-born academic scientists , 2018, Science and Public Policy.

[61]  Henk F. Moed,et al.  Studying scientific migration in Scopus , 2013, Scientometrics.

[62]  Rodrigo Costas,et al.  Interpreting "altmetrics": viewing acts on social media through the lens of citation and social theories , 2015, ArXiv.

[63]  Jorge E. Hirsch,et al.  hα: An index to quantify an individual’s scientific leadership , 2018, Scientometrics.

[64]  Nicolás Robinson-García,et al.  Making sense of global collaboration dynamics: Developing a methodological framework to study (dis)similarities between country disciplinary profiles and choice of collaboration partners , 2019, ISSI.

[65]  N. Robinson-García,et al.  The Global Scientific Workforce (GTEC) Framework , 2018 .

[66]  Evaristo Jiménez-Contreras,et al.  Analyzing the disciplinary focus of universities: Can rankings be a one-size-fits-all? , 2017, ArXiv.

[67]  Carolina Cañibano,et al.  International Stays Abroad, Collaborations and the Return of Spanish Researchers , 2015 .

[68]  Ismael Rafols,et al.  How journal rankings can suppress interdisciplinarity. The case of innovation studies in business and management , 2011, ArXiv.

[69]  Vincent Larivière,et al.  On the development of China’s leadership in international collaborations , 2019, Scientometrics.

[70]  Jacqueline Leta,et al.  Recognition and international collaboration: the Brazilian case , 2002, Scientometrics.

[71]  Susan B. Twombly,et al.  International Faculty in American Universities: Experiences of Academic Life, Productivity, and Career Mobility , 2012 .

[72]  D. Hicks The Four Literatures of Social Science , 2004 .

[73]  David Adam,et al.  Citation analysis: The counting house , 2002, Nature.

[74]  Emilio Delgado López-Cózar,et al.  The evolution of research activity in Spain , 2003, Research Policy.

[75]  Vincent Larivière,et al.  Scientists have most impact when they're free to move , 2017, Nature.

[76]  M. J. Martín-Sempere,et al.  Generating favourable contexts for translational research through the incorporation of basic researchers into hospitals: The FIS/Miguel Servet Research Contract Programme , 2012 .

[77]  Henk F. Moed,et al.  Citation Analysis in Research Evaluation , 1899 .

[78]  Fei Shu,et al.  Publish or impoverish: An investigation of the monetary reward system of science in China (1999-2016) , 2017, Aslib J. Inf. Manag..

[79]  Carolina Cañibano,et al.  Gender and patterns of temporary mobility among researchers , 2016 .

[80]  Ismael Rafols,et al.  Indicators of the interdisciplinarity of journals: Diversity, centrality, and citations , 2010, J. Informetrics.

[81]  Izabela Zych,et al.  How to measure the internationality of scientific publications. , 2012, Psicothema.

[82]  Henk F. Moed,et al.  Improving the accuracy of Institute for Scientific Information's journal impact factors , 1995 .

[83]  Brian Hayes,et al.  Counting House , 2019, Chaucer.

[84]  José Luis Ortega,et al.  Comparing university rankings , 2010, Scientometrics.

[85]  Henk F. Moed,et al.  Multidimensional assessment of scholarly research impact , 2014, J. Assoc. Inf. Sci. Technol..

[86]  N. Robinson-García,et al.  Using Altmetrics for Contextualised Mapping of Societal Impact: From Hits to Networks , 2017 .

[87]  A. Henshall English language policies in scientific journals: Signs of change in the field of economics , 2018, Journal of English for Academic Purposes.

[88]  A. Elzinga The New Production of Knowledge. The Dynamics of Science and Research in Contemporary Societies , 1997 .

[89]  Teaching workload and satisfaction of foreign-born and U.S.-born faculty at four-year postsecondary institutions in the United States. , 2011 .

[90]  T. V. Leeuwen,et al.  Impact factors can mislead , 1996, Nature.

[91]  Cassidy R. Sugimoto,et al.  Travel bans and scientific mobility: utility of asymmetry and affinity indexes to inform science policy , 2018, Scientometrics.

[92]  N. Robinson-García,et al.  From theory to practice: Operationalization of the GTEC framework , 2018 .

[93]  Lili Wang,et al.  Determinants of citation impact: A comparative analysis of the Global South versus the Global North , 2017 .

[94]  Philip G. Altbach,et al.  Globalisation and the University: Myths and Realities in an Unequal World , 2004 .