Without blinders on, academia offers a lot more perspective

The developments surrounding Recognition and Rewards have caused quite a stir this summer. The central question is how the university should make decisions when selecting, evaluating and rewarding researchers. In this debate, which took  place both nationally and internationally, a lot of attention was paid to the use of the Journal Impact Factor (JIF) as a measure of the quality of individual publications and researchers. Utrecht University (UU) has decided to no longer use the impact factor for selecting and promoting academic employees. Although this plan has received a lot of attention, it is only a small part of our university’s broader vision on Recognition and Rewards.

This vision is centered around broadening the scope of evaluation: from (mostly) research and (a little) education to an equal balance of education, research and professional activities (such as patient care). Moreover, internationally, nationally, and at UU, attention is shifting from the individual to the collective. There will be more recognition for leadership and teamwork, and more attention for the actual impact of research and education.

Working with blinders on prevents a pleasant working environment

This vision also means that teams (such as education teams or research groups) will have the space to determine their own strategies and goals. Teams will get to deliberate more about what they want to achieve and the competencies within the team necessary to accomplish those goals. This creates room for diversity in the tasks and competencies. In other words: not everyone in a team will have to be good at the same thing, as not everyone will have to be able to do everything.

This does not amount to throwing quality out of the window, but rather developing a broader idea of what quality means. No longer narrowly focusing on a couple of indicators, and instead looking around at what moves us, university colleagues. Working with blinders on prevents a pleasant working environment and makes us lose sight of what constitutes good education and research with societal worth.

Impact factors do not make for nice colleagues
According to some, there is no alternative to impact factors, and deciding to forego them is bad for science. The apparent objectivity of the impact factor and the h-index has seduced many academics and administrators, with some fields having become so used to them that it seems difficult to let them go.

Using reputations as a short-cut for appraisal is enticing. But reputation is self-reinforcing, and that is problematic. In the current system, mostly authors of prestigious universities get to publish in journals with a high impact factor. The authors know that this will give them more visibility. At the same time, editors are aware that the noteworthiness of the authors and their university will lead to more citations and an eventual increase in the journal’s impact factor. In turn, researchers feel obligated to cite these prestigious journals lest their own work looks unimportant or they risk treading on the toes of reviewers. Meanwhile, prestigious journals hamper the movement towards open access (or ask for extremely large Article Processing Charges) to protect their profits. Impact factors are like blinders: the jockey puts them on the horse, and thereby perpetuates existing privileges.

The first question is not: ‘what is it about?’ but ‘how high is the impact factor?’

Aside from the fact that impact factors are everything but objective criteria for research quality and suffer from considerable methodological issues, they do not contribute to making academia a nicer place to work.

Many of us have experienced that while telling a colleague about a new publication over coffee, the first question is not ‘what is it about?’ but rather ‘how high is the impact factor?’. PhD candidates invest blood, sweat and tears in articles they are then expected to take down the cascade of journals (by impact factor) ad nauseum, continually adjusting their article along the way. Not in the name of science, but in the name of the impact factor.

What now?
All the same, there are still those who think that impact factors will remain necessary in the future to rightly appraise science. But if you truly pause to think about what kinds of new possibilities this new movement offers, then using impact factors or not is only a small part of the greater vision of Recognition and Rewards.

Because what is it all about? Our university’s ability to contribute to society. That is possible through research teams who can build on the efforts of colleagues from different institutes to develop a vaccine. But also through teaching Economics students about ethics, or Dutch-language students about artificial intelligence. Or by inspiring a single student who is experiencing a dip in motivation. Or even through not expecting colleagues to put their hobbies or care duties at home aside so that they can devote themselves exclusively to science outside of working hours.

We should not expect colleagues to put aside their hobbies so that they can devote their evenings exclusively to science

At our ideal university, colleagues and managers learn about each other’s work. Their opinion of a researcher is not, due to a lack of time, based on impact factors or the obtainment of research grants that sometimes resemble lotteries. Paying attention to the work itself of course necessitates an adequate financing for higher education and an appreciation of teamwork and leadership.

During department meetings, people also talk about their teaching or participation in public debate. And when it is about research, the primary concern is not how to be published in as prestigious a journal as possible, but rather: how can we best spread this idea? With which societal partners can we continue thinking about it? How can we help this young colleague to set up a blog or publish an opinion piece about it?

By stepping away from a one-sided emphasis on publications in our evaluations (especially publications in so-called ‘top journals’ or with prestigious publishers), we make space for all the other things that make academia so valuable. Education, collaboration, guidance and a more open academic culture – with attention for different research questions (that might not be immediately considered for Science or Nature), reproducibility, sharing code and data, and public engagement.

What’s next?
A new Recognition and Rewards system will offer opportunities to do away with the negative effects of competition and foster an open academic culture focused on collaboration. This will not only benefit personnel and students, but also science itself, and the relationship between science and society.

In the coming months, UU will further develop the new Recognition and Rewards vision, by working on a template that the faculties can get started with. All the UU staff will be expressly involved.

Let’s put away our blinders and look around. Recognizing and rewarding differently is possible, and will strengthen our autonomy and team spirit. We will be free to define for ourselves what quality we aim to achieve, without putting stock in bad, one-sided metrics.