Hans Clevers: new Recognition & Reward system will have "disastrous" consequences

“My university has discarded all quantitative parameters for publishing performance”, Clevers tweeted in December. “Prediction: we will fall out of the international rankings. Foreign students will stay away. Work pressure will decrease. Problem solved.”

With this cynical slant, the professor and research group leader of Utrecht’s Hubrecht Institute and the Princess Máxima Centre for Child Oncology expressed his discontent with the new criteria for recognising and rewarding science that UU and other universities now use in their tenure and promotion policies.

In that new policy, focus has been shifted away from the number of published studies in top journals and their citations. Journal impact factors and h-indexes have little bearing on the quality of researchers, and may even encourage perverse incentives. That is why an increasing number of university board members think it would be better if scientists would show their ability to work with the requirements of ‘Open Science’, such as promoting teamwork and publishing in open access journals.

It will become an essay competition

Clevers concludes that Dutch universities will lower their scientific ambitions to remedy the problems that have arisen due to chronic underfunding. A disastrous road, in his view, especially now that science financier NWO has embraced the new Recognition and Reward system.

“I am very concerned about this”, the professor says. “When applying for a grant in the Netherlands you can no longer say what you have published or done before. You now need to provide a ‘narrative’, a story about the importance of your research. It will become an essay competition – bureaus that will help with the writing are lying in wait.”

One of the things he is worried about is the future of young laboratory researchers. “They are so anxious. They have chosen to follow their passion and really want to do science at the highest international level. But their chances of reaching and staying at that level by receiving NWO grants are getting ever smaller. And if they do get the grant, they can’t publish their results in Nature of Science.”

That is why he is now sounding the alarm. “In ten years, I don’t want anyone to accuse me of not speaking up. If this continues, it will be disastrous.”

That tone – as if a Nature publication was wrong by definition…

The professor stresses the importance of what he calls the “output measurement”. Clevers: “Just like a car salesperson is judged based on the amount of cars they have sold, researchers should be able to be evaluated based on their discoveries. We must be able to do that, I think. That is what science is about, at its core.”

The professor concedes that the way that those research efforts have been measured up until now may not be ideal, but he thinks they are not that bad. He denounces the way Dutch policymakers have deemed publishing in top journals as “bravado and machismo”.

“Nowhere else in the world do they talk so negatively about those journals. That tone – as if a Nature publication is wrong by definition... If that were true, then Nature wouldn’t even exist. All researchers send their best work to those journals, as that is where the toughest peer reviewers and the most professional editors are. I don’t understand how that way of judging quality has become unimportant all of a sudden. Not to mention that those journals are the most read. If something is not right, it’s bound to come out there. Other journals publish a lot more shit.”

Many of his international colleagues uttered their amazement when UU professor Paul Boselie wrote an editorial for Nature about how the university was going to say goodbye to quantitative performance measurements. “They are quite giggly about it: doesn’t UU care about good science?

“Of course there are lots of disadvantages, but not having any consideration for impact and rankings? Sorry, but people look at them everywhere else. In the Netherlands, universities think that foreign countries will follow. That sounds a tad arrogant to me.”

Honestly, I don’t know any open access journals with high impact

Clevers suspects the resistance to top journals originates from their business model, as they charge high subscription fees for publishing research that has been executed with public funds, and commercial publishers turn high profits. Clevers understands the critique, but the proposed solution, namely publishing in open access journals, still has too many shortcomings.

“To have research impact, journals need to reject 90 to 95 per cent of the articles submitted to them, and accept only the strongest and most sensational ones. That doesn’t work for open access journals, as their business model dictates the acceptance of 80 to 90 percent of submitted articles. That is the only way they can turn a profit: the publishing author is the only source of income and often pays 4,000 to 5,000 Euros per article. The bar is low, and honestly I don’t know any open access journals with high impact.”

“Scientists who take themselves seriously submit their articles to one of the top journals first. If they get rejected there, then they move on to a ‘lower’ journal. I agree that, ideally, everything should be open access. Scientists would love nothing more than the world being able to access their findings. But let’s not pretend that journals that aren't read and cited much are more important than journals with high impact. It would be better for us to demand that Nature and Science become open access!”

The former Royal Netherlands Academy of Arts and Sciences director thinks the emergence of the new Recognition & Reward system has also arisen from the plight that universities have found themselves in due to insufficient government funding. Public financing for scientific research has been lacking in the Netherlands for years.

“The workload is high, people have many responsibilities and there is not enough money. Young researchers are in the worst situation of all. It’s logical to not only base performance evaluation on research, but on education, management work and outreach too. After all, all of that is just as important. It’s true that universities have let themselves be led by research performance alone in their employee policies for far too long", says the professor. “But to then respond with ‘we’ll stop using quantitative output parameters in research, but judge people solely by how they work and not what they have discovered at the end of the day’... That does not seem like the right way to me." 

“I think Recognition & Reward overlooks the fact that top research belongs in a healthy scientific landscape, and that there should be space for young people who know that they have the talent and passion to become international luminaries. Perhaps a university can choose to exempt maybe ten per cent of its research staff for truly top science. But the bar should be put high for these scientists, and we should be able to measure their performance", he suggests. “And of course: collaboration is important for a researcher, but there are more things, such as leadership, being a mentor and a good citizen, to name but a few. Those are difficult to objectify.”

Juries are kept in the dark

That research funder NWO will use the new Recognition & Reward criteria is even worse, in Clevers' point of view. “Universities are responsible for more than science, but the NWO is undermining its own raison d’être, which is attributing funds to research proposals in an objective competition driven by external peer review. That is so beautifully complementary to what universities do, as projects are not tested though peer review beforehand, but afterwards.”

Both on Twitter and in an article in the journal Thema, which is aimed at higher education professionals, Clevers describes what Dutch science was like when he started his career. Due to a lack of output or publishing demands, Dutch higher education was a boring affair. Clevers thinks that the introduction of external evaluation, the open competition for NWO funds and output parameters are the reason Dutch science is now among the best in the world.

“The combination of universities that are able to provide secure positions to staff and the open competition for NWO funds with external peer review is ideal. Why would you want to change that? Of course we do need enough money to fund the system.”

Visibly annoyed, Clevers enumerates some examples of fellow scientists who has their NWO applications sent back because they were not allowed to refer their scientific track record. “That is so bad. Juries are kept completely in the dark. It’s a kind of raffle.”

With Recognition & Reward, Dutch science is running the risk of returning to the same situation it was in the 1970s, says Clevers. “If you let everyone participate, if it’s about playing the game rather than winning, that would be very pleasant for a lot of people. But it would also make the Dutch system go out of business.”