Does the Rubicon grant really help junior researchers?
With a Rubicon grant, junior researchers can embark on an international adventure. Photo: Pixabay
The Rubicon grant was established in 2005. Since then, it has become quite popular and acquired a good reputation. However, it is rather difficult to measure their impact, says Marielle Non, from CPB.
The NWO selection committees rank the applicants from the strongest to the weakest. As part of a study commissioned by CPB, Non analysed the applicants who ranked in the middle, just above or below the cut-off. That means the ones who just made it or just missed it.
Effects
There’s no simple metric to compare all the applicants, Non pointed out in an online audience about the study. It may well be that the best researchers would have had successful careers anyway, while the weakest wouldn’t go so far. In order to isolate the grant's effect, one needs a basis to compare the researchers.
Surprisingly enough, for those ranked around the cut-off, there is hardly any discernible difference between the four winners and the four losers. There are no discernible differences with regard to how much they publish after receiving their grant, the impact of their articles, or whether they stop publishing scientific papers altogether.
Sometimes the outcomes seem to weigh a bit more in favour of the winners, but not enough. NWO committees have no trouble selecting the top candidates, but for those in the middle the choice is somewhat arbitrary. In her study for CPB, Non checked her outcomes in all kinds of ways, but no matter what she tried, the differences remained negligible.
Can't measure it all
But some things just can’t be measured. For some individual researchers, the grant may be extremely important. Moreover, you can't say that an international experience is not meaningful just because the best researchers would have had successful careers anyway.
On the other hand, Non's findings are consistent with the plea to introduce a lottery system to decide on applicants who rank closely together. That might feel more justified than a rejection which isn’t really based on anything, one of the people who attended the online presentation observed.
Another audience member raised practical objections. A lottery system would require the committee to use two cut-offs: one for those who absolutely should be selected and one for those who should clearly be dropped. That would mean a lot of additional work for results that would not be significantly different, not to mention it probably wouldn’t put an end to the discussion about how to evaluate researchers just above or just below the cut-off.