Coherence between policy formulation and implementation of public research support? An examination of project selection mechanisms in the Norwegian Research Council

In this study a unique dataset covering different aspects of official evaluators’ ex ante evaluation of Norwegian private firms’ research support applications, in addition to objective ex post short term success indicators, has been made accessible for empirical scrutiny. The empirical study is conducted in two stages, following a Heckman correction procedure (Heckman, 1979). In the first stage, data from all projects that applied for UIOP-support in the period 2000 – 2008 are used to examine how ex ante evaluation criteria scores influence the probability of being selected for support. The predicted probabilities for each of the projects are used to calculate the Inverse Mills Ratio (IMR), which will be used in the second stage to overcome possible selection bias problems. In the second stage the relationship between the ex ante evaluation scores and a set of different objective measures of ex post spillover effects of the supported projects are estimated by a set of negative binomial regressions. The dependent variables in this stage are measured as counts of different spillover scores, ranging from zero to many, and the IMR from the first stage is included as one of the explanatory variables. There is to some extent coherence between ex ante evaluations of spillover effects and ex post short term knowledge diffusion effects. However, there seems to be a general lack of coherence between ex ante evaluations and ex post spillover effects connected to technology diffusion. It is also interesting to note that, on one hand, there is a significant negative relationship between the monetary amount applied for and the likelihood of being selected, indicating that small projects are preferred. On the other hand, it is also evident that larger projects produce more ex post spillover effects both in terms of knowledge and technology diffusion.