A concept for inferring ‘frontier research’ in grant proposals

This paper discusses a concept for inferring attributes of ‘frontier research’ in peer-reviewed research proposals under the popular scheme of the European Research Council (ERC). The concept serves two purposes: firstly to conceptualize, define and operationalize in scientometric terms attributes of frontier research; and secondly to build and compare outcomes of a statistical model with the review decision in order to obtain further insight and reflect upon the influence of frontier research in the peer-review process. To this end, indicators across scientific disciplines and in accord with the strategic definition of frontier research by the ERC are elaborated, exploiting textual proposal information and other scientometric data of grant applicants. Subsequently, a suitable model is formulated to measure ex-post the influence of attributes of frontier research on the decision probability of a proposal to be accepted. We present first empirical data as proof of concept for inferring frontier research in grant proposals. Ultimately the concept is aiming at advancing the methodology to deliver signals for monitoring the effectiveness of peer-review processes.

[1]  Claire François,et al.  Detecting domain dynamics: Association Rule Extraction and diachronic clustering techniques in support of expertise , 2011 .

[2]  Claire François,et al.  Identification and characterisation of technological topics in the field of Molecular Biology , 2010, Scientometrics.

[3]  Michel Zitt,et al.  Challenges for scientometric indicators: data demining, knowledge-flow measurements and diversity issues , 2008 .

[4]  Alain Lelu Modeles neuronaux pour l'analyse de donnees documentaires et textuelles : organiser de très grands tableaux de données qualitatives en pôles et zones d'influence , 1993 .

[5]  Anton J. Nederhof,et al.  Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review , 2006, Scientometrics.

[6]  B. M. Gupta,et al.  Analysis of distribution of the age of citations in theoretical population genetics , 1997, Scientometrics.

[7]  M. Hojat,et al.  Impartial Judgment by the “Gatekeepers” of Science: Fallibility and Accountability in the Peer Review Process , 2003, Advances in health sciences education : theory and practice.

[8]  David Adam,et al.  Citation analysis: The counting house , 2002, Nature.

[9]  Primoz Juznic,et al.  Scientometric indicators: peer-review, bibliometric methods and conflict of interests , 2010, Scientometrics.

[10]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[11]  Claire François,et al.  STANALYST: An integrated environment for clustering and mapping analysis on science and technology , 2001 .

[12]  Claire François,et al.  Hypertext paradigm in the field of information retrieval: a neural approach , 1993, ECHT '92.

[13]  Bo Jarneving,et al.  Bibliographic coupling and its application to research-front and other core documents , 2007, J. Informetrics.

[14]  Loet Leydesdorff,et al.  Past performance, peer review and project selection: a case study in the social and behavioral sciences , 2009, 0911.1306.

[15]  Kevin W. Boyack,et al.  * Sandia Is a Multiprogram Laboratory Operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under Contract De-ac04-94al85000. Quantitative Evaluation of Large Maps of Science , 2022 .

[16]  L. Bornmann,et al.  The effectiveness of the peer review process: inter-referee agreement and predictive validity of manuscript refereeing at Angewandte Chemie. , 2008, Angewandte Chemie.

[17]  L. Egghe,et al.  Theory and practise of the g-index , 2006, Scientometrics.

[18]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[19]  Yang Tao,et al.  A Study on Development Planning for Management Science and Engineering , 2006 .

[20]  Wolfgang Glänzel,et al.  A bibliometric study on ageing and reception processes of scientific literature , 1995, J. Inf. Sci..

[21]  Sungjoo Lee,et al.  Development and application of a keyword-based knowledge map for effective R&D planning , 2010, Scientometrics.

[22]  Chaomei Chen,et al.  Measuring the movement of a research paradigm , 2005, IS&T/SPIE Electronic Imaging.

[23]  Simon Parsons,et al.  Principles of Data Mining by David J. Hand, Heikki Mannila and Padhraic Smyth, MIT Press, 546 pp., £34.50, ISBN 0-262-08290-X , 2004, The Knowledge Engineering Review.

[24]  Claire François,et al.  An advanced diffusion model to identify emergent research issues: the case of optoelectronic devices , 2010, Scientometrics.

[25]  Kevin W. Boyack,et al.  Co-citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately? , 2010, J. Assoc. Inf. Sci. Technol..

[26]  Henry G. Small,et al.  Co-citation in the scientific literature: A new measure of the relationship between two documents , 1973, J. Am. Soc. Inf. Sci..

[27]  R. Rousseau,et al.  The R- and AR-indices: Complementing the h-index , 2007 .

[28]  K. Viljamaa,et al.  Emergence of Economic Institutions: Analysing the Third Role of Universities in Turku, Finland , 2008 .

[29]  H. Marsh,et al.  Improving the Peer-review Process for Grant Applications , 2022 .

[30]  Naoki Shibata,et al.  Comparative study on methods of detecting research fronts using different types of citation , 2009, J. Assoc. Inf. Sci. Technol..

[31]  Fawzy A. Torkey,et al.  A Text Mining Technique Using Association Rules Extraction , 2008 .

[32]  B. Sweitzer,et al.  How well does a journal's peer review process function? A survey of authors' opinions. , 1994, JAMA.

[33]  Richard Van Noorden,et al.  Metrics: A profusion of measures. , 2010, Nature.

[34]  魏屹东,et al.  Scientometrics , 2018, Encyclopedia of Big Data.

[35]  Heikki Mannila,et al.  Principles of Data Mining , 2001, Undergraduate Topics in Computer Science.