Drawing straws for research grants: Lousy or savvy?

In a press release, UMC Utrecht described the occurrence as “a novelty”. That sounds a little over the top, but, in any case, it is remarkable that the starter and incentive grants created by the Minister of Education, Robbert Dijkgraaf, worth 2.5 million euros, were distributed by drawing straws. 

Five starter grants were available, for which eleven divisions could nominate two young assistant professors each, provided that they met the requirements. For the six stimulation grants available, all other lecturer-researchers were eligible. They just had to submit a one-page proposal to be considered. A total of 192 proposals were submitted in the end. 

For UMC Utrecht, this step makes sense. After all, the grants were meant to give lecturer-researchers more "room" to do their work "in peace" and not as a competition to see who has the best research plan. The minister would have liked to give everyone a grant, but there weren't enough funds to do so. 

The university hospital then wondered why it would set up a selection committee and a time-consuming assessment process. By choosing a lottery system, UMC Utrecht reawakened an old discussion in the Netherlands: are draws a good way to distribute grants in national and international competitions?

Best-case scenario
Femke van Wijk, Professor of Translational Immunology, was one of the members of the committee that proposed the lottery system. She acknowledges that not everyone at the hospital was excited about the idea. “Scientists like to measure things and often think they can come up with objective standards. They feel that a lottery is a sign of weakness. However, when we asked them to come up with a feasible alternative, we only got silence.”

Van Wijk sees the lottery as the best-case scenario. “Everyone eligible for these grants already has tenure, which means that UMC Utrecht already considers them competent scientists.”

Not the first time
This case is far from being a first. UMC Utrecht itself had already distributed grants for children’s research that way and Tilburg University draws straws sometimes as well. Last year, Tilburg's university magazine Univers reported on a meeting where small research grants were divided among Humanities researchers.

The Centre for Unusual Collaborations (Cuco), which encourages collaboration between scientists from the universities of Eindhoven, Wageningen and Utrecht, has distributed one million euros worth of research grants through a lottery system for the past two years. Cuco supported seven research teams that way in 2023 and another six in 2024. 

According to UMC epidemiologist Inge Stegeman, who also serves as the chair of Cuco's board, the lottery system was chosen to eliminate as many obstacles in the way of researchers interested in interdisciplinary work as possible. After all, it's often difficult to find regular funding for complex projects that use a myriad of research methods. “We are looking to encourage such collaborations. One shouldn’t exhaust researchers with elaborate proposal and selection procedures, as that can kill their ideas.”

At Cuco, all proposals are put in small balls in a big goblet. Those eligible are invited to be present live or online. “The first time, I thought it might be a fun party. But there are always losers, so it’s inappropriate to announce the winners with a drum roll and festivities. However, I believe it’s good to be transparent about it,” says Stegeman.

Pressure
The examples from UMC Utrecht and Cuco seem to demonstrate that lottery systems can be a good way to distribute research funds. Van Wijk and Stegeman acknowledge that they are dealing with very specific grants, but they believe it is worth investigating whether larger, more competitive funding programmes could also include lottery systems as a means to divide funds in a fairer, more efficient, and rational manner. 

In her oration, given a year and a half ago, Femke van Wijk voiced concerns about the emotional damage researchers can suffer as a result of working in an environment that focuses too much on grants and achievement indicators. In her view, academia does not benefit from that. “In academia, we act as though we’re capable of deciding who the most talented and promising researchers are. Those are the ones who obtain the grants, of course. If you manage to obtain a part of the scarce funds, that’s seen as a huge achievement. You managed to stand out from all the people who did not get a grant. In reality, however, that thought process doesn’t hold up.”

According to Van Wijk, scientific literature on talent development shows that it’s hard to identify which researchers will be successful. One thing for which there is proof is the so-called Mattheus effect: the more someone obtains success in academia, the more likely they will become to obtain more funds and status. 

Frustration
The professor sees a lot of competition fatigue around her. “Applications are becoming bigger and more complex, and people are getting more and more frustrated, often because they don’t understand much about why review committees reject their applications. That makes sense. After all, those committees must make distinctions when all applications are all of high quality.”

In Van Wijk's opinion, "conditioned lotteries" might offer a solution. In this system, excellent applications are accepted, abominable ones are ignored, and straws are drawn for all the others. 

“Selection committees often agree pretty quickly about which applications definitely deserve funding and which just won’t cut it. But the others are almost impossible to decide on, they’re all good. Then, preconceptions and personal issues end playing a role: ‘Oh, I know that one, they’re really good’.”

Part of Open Science 
Inge Stegeman is an epidemiologist and works at UMC researching how the quality of biomedical research can be strengthened. She sees the ambition for better funding methods as part of a new mentality surrounding the recognition and rewarding of scientists, as well as a strive for Open Science. 

“No one is saying that lotteries are an end-all solution at all times, but Cuco is a great space to test things and see what works. We really want to study the effects of our lottery system, also in the long run. Is it truly a better, fairer method that leads to more successful projects? How do researchers experience it? We don’t know that yet.”

Not deaf
Van Wijk and Stegeman’s ideas are not new. It's a debate that has been going on in the Netherlands for almost twenty years. Fifteen years ago (link in Dutch), Rathenau Institute denounced the "randomness" with which the Dutch Research Council (NWO) made distinctions between researchers who are "good".

Five years later, UU professor Ingrid Robeyns suggested (also in Dutch, Ed.) that a lottery system could end that "farce". Two years ago, scientists in Amsterdam wrote that a partial lottery has the potential to make funding fairer, more efficient, and more diverse. 

Research financers were not deaf to the increasing call for less pressure on grant applications, less time-consuming procedures, and more transparency. NWO announced a goal of cutting procedure time by 20 percent, as well as reducing workloads for researchers, referents and committee members by 25 percent. Because of that, the Veni, Vidi and Vici grants now work with pre-applications and the consultation of referents or holding interviews could be dropped. 

NWO boss Marcel Levi is not convinced of the value of lottery systems, however. In an interview with the Higher Education Press Agency (HOP), he said that would be a lazy solution. He also participated in a debate last year, in which he stated that his most important task is to increase the acceptance rates to a quarter and then ensure that the best quarter of applications gets accepted.

Still, NWO says it has been following lottery experiments with great interest. A spokesperson refers to a report from the Research on Research Institute, which describes examples of "partial randomizing" (painstakingly avoiding the term "lottery") from New Zealand, Switzerland and Austria.

Arguments against it
In addition to the aforementioned arguments in favour of lottery systems, the report also mentions arguments against it. Researchers might work less hard on the quality of their proposals, for example. In addition, researchers who obtain a grant through a lottery system might get less recognition, and science itself might have its reputation damaged once it becomes clear that academics themselves can’t tell good or bad science apart. 

There are many voices against drawing straws at Utrecht University, if one listens hard enough. UU Biology Professor Sander van den Heuvel sympathises with NWO president Levi’s reluctance, for instance. Though he acknowledges many snags in the current research funding system, he calls lotteries "lousy". In his view, one should be able to "ensure  there are clear reasons for granting or not granting funds to a study or researcher.”

Van den Heuvel doesn't judge UMC Utrecht’s decision to distribute its starter and incentive grants through a lottery system as the decision isn't as hard to make in his research group as it is in the hospital. “We’re able to hire PhD candidates for all starting assistant professors who combine research and teaching. We use the starters and stimulation grants for that.”

He believes that the low acceptance rates and lack of time for referents and committee members are the biggest issues to be fixed when it comes to national competitions for grants. For this reason, he applauds the debate about better methods of distributing research funding. “You could say that it’s basically a lottery ticket if you are given a grant now. That’s not how it should be.”

He is less enthusiastic about some of the time-saving innovations. “Interviews with candidates have been dropped in the Life Sciences part of the Vidi procedure. In my opinion, they're losing an essential element.” He’s more positive about an initiative for the so-called XS grants for innovative research, which are smaller. Scientists who submit a grant application there also agree to assess other people's applications. 

Maintaining the stimulus
Rose Masereeuw, Vice Dean at the Faculty of Science, isn’t all that excited about lottery systems either. She writes: “I think lottery systems are bad, I don’t see how they could improve the quality of research.”

She underscores how valuable the experience of developing research plans and receiving feedback from colleagues can be for young researchers. In her view, procedures should aim to honour the best proposals. 

“That doesn’t mean the ones who don’t make it aren’t good. We often see projects make it through other means, following the feedback and input provided by the competition. I believe the stimulus for creating a high-quality research proposal is taken away when you adopt a lottery system.”

Inge Stegeman calls for nuance in this debate. “Maybe the quality of proposals will decrease if we introduce a lottery system, but might also increase. It is more likely we will have both upsides and downsides. We shouldn’t talk about it like it’s black and white. Besides, it’s pretty hard to decide whether you’re for lottery systems or against them because we don’t know much about their effects yet. We need long-term research for that, so first we need to run pilots.”