The acceptability of using a lottery to allocate research funding: a survey of applicants

Background The Health Research Council of New Zealand is the first major government funding agency to use a lottery to allocate research funding for their Explorer Grant scheme. This is a somewhat controversial approach because, despite the documented problems of peer review, many researchers believe that funding should be allocated solely using peer review, and peer review is used almost ubiquitously by funding agencies around the world. Given the rarity of alternative funding schemes, there is interest in hearing from the first cohort of researchers to ever experience a lottery. Additionally, the Health Research Council of New Zealand wanted to hear from applicants about the acceptability of the randomisation process and anonymity of applicants. Methods This paper presents the results of a survey of Health Research Council applicants from 2013 to 2019. The survey asked about the acceptability of using a lottery and if the lottery meant researchers took a different approach to their application. Results The overall response rate was 39% (126 of 325 invites), with 30% (76 of 251) from applicants in the years 2013 to 2018, and 68% (50 of 74) for those in the year 2019 who were not aware of the funding result. There was agreement that randomisation is an acceptable method for allocating Explorer Grant funds with 63% ( n = 79) in favour and 25% ( n = 32) against. There was less support for allocating funds randomly for other grant types with only 40% ( n = 50) in favour and 37% ( n = 46) against. Support for a lottery was higher amongst those that had won funding. Multiple respondents stated that they supported a lottery when ineligible applications had been excluded and outstanding applications funded, so that the remaining applications were truly equal. Most applicants reported that the lottery did not change the time they spent preparing their application. Conclusions The Health Research Council’s experience through the Explorer Grant scheme supports further uptake of a modified lottery.

[1]  Funding Science by Lottery , 2015 .

[2]  Nicholas Graves,et al.  Funding grant proposals for scientific research: retrospective analysis of scores by members of grant review panel , 2011, BMJ : British Medical Journal.

[3]  Mitchell J. Nathan,et al.  Low agreement among reviewers evaluating the same NIH grant applications , 2018, Proceedings of the National Academy of Sciences.

[4]  Gunther Eysenbach,et al.  Improving the Quality of Web Surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES) , 2004, Journal of medical Internet research.

[5]  Luigi Salmaso,et al.  Permutation Tests for Complex Data , 2010 .

[6]  D. Adam.  Science funders gamble on grant lotteries , 2019, Nature.

[7]  Nicholas Graves,et al.  On the time spent preparing grant proposals: an observational study of Australian researchers , 2013, BMJ Open.

[8]  S. Avin.  Mavericks and lotteries. , 2019, Studies in history and philosophy of science.

[9]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[10]  A. Casadevall,et al.  NIH peer review percentile scores are poorly predictive of grant productivity , 2016, eLife.

[11]  B. Edlund.  Unconscious Bias. , 2016, Journal of gerontological nursing.

[12]  Arturo Casadevall,et al.  Grant funding: Playing the odds. , 2016, Science.

[13]  N. Graves,et al.  The impact of a streamlined funding application process on application time: two cross-sectional surveys of Australian researchers , 2015, BMJ Open.

[14]  Arturo Casadevall,et al.  Research Funding: the Case for a Modified Lottery , 2016, mBio.

[15]  V. Demicheli,et al.  Peer review for improving the quality of grant applications. , 2007, The Cochrane database of systematic reviews.

[16]  JOHN EVANS,et al.  The Royal Society , 1894, Nature.

[17]  N. Graves,et al.  The research lottery : the pressures on the Australian grant system , 2014 .

[18]  Sara Schroter,et al.  Surveys of current status in biomedical science grant review: funding organisations' and grant reviewers' perspectives , 2010, BMC medicine.

[19]  A. Barnett,et al.  Funding by Lottery: Political Problems and Research Opportunities , 2016, mBio.

[20]  N. Graves,et al.  ‘Are you siding with a personality or the grant proposal?’: observations on how peer review panels function , 2017, Research Integrity and Peer Review.

[21]  Lambros Roumbanis.  Peer Review or Lottery? A Critical Analysis of Two Different Forms of Decision-making Mechanisms for Allocation of Research Grants , 2019, Science, Technology, & Human Values.

[22]  Steven Wooding,et al.  What do we know about grant peer review in the health sciences? , 2017, F1000Research.

[23]  Ohid Yaqub,et al.  Serendipity: Towards a Taxonomy and a Theory , 2016 .

[24]  Carl T. Bergstrom,et al.  Contest models highlight inherent inefficiencies of scientific funding competitions , 2018, PLoS biology.

[25]  Nicholas Graves,et al.  The impact of funding deadlines on personal workloads, stress and family relationships: a qualitative study of Australian researchers , 2014, BMJ Open.