'Frustrating' assessment of PhD's disappears; more attention for own development

In grand café Het Gegeven Paard, below music centre TivoliVredenburg, we easily recognise PhD candidate and future neurologist Annemijn Algra by her striking, pink-purple jacket. It had once been meant as a one-time gimmick, but these days, she always wears the jacket when she talks about her work for Young Science in Transition, a thinktank for young researchers.

Algra was at the basis of a new way of assessing the PhD candidates at the Graduate School of Life Sciences at Utrecht University. Instead of the number of publications, it now focuses on the development and ambitions of the PhD candidate. A much more sensible way to evaluate people, she says. More than two hundred PhD candidates are now working with the adjusted assessment criteria; eventually, all 1,800 PhD candidates and their supervisors are to use them.

A PhD candidate who, from a group of critical young scientists, manages to shift the conservative academic and medical worlds: that causes quite the stir. But there’s another reason why the new evaluation process for PhD candidates’ work attracts a lot of attention.

Universities can’t stop talking about the importance of new ways of recognising and evaluating: moving away from the idea of the scientist as a superstar and jack of all trades; more attention to specific individual qualities, different career paths, and team science. Those same universities have been struggling to be specific about what that change will actually mean in practice. Algra, however, can do what the universities haven’t – at least for the UU PhD candidates in life sciences.

She talks about it with justifiable pride. “If you’re tenacious enough, you can really achieve something, apparently.”

What do you think was wrong with the way supervisors looked at the work of PhD candidates?
“It was literally the case that every year, you’d have to submit a Word document in which the first question asked you to indicate how many articles you’d published. That’s frustrating and increases stress, especially for starting PhD candidates. They have to fill out ‘zero’ everywhere, because they’ve only been working on setting up their projects.

“And everyone knows the minimum requirement: four articles for a PhD. I was never able to find where that number came from, but it’s a standard that’s still used very often. Not just by supervisors; PhD candidates rile up each other as well, perhaps even more so.

‘PhD candidates rile each other up’

“Moreover, your mentor would be present at that annual chat with your supervisor and co-supervisors. That was well-intentioned, trying to safeguard a safe learning environment. But what do you think a PhD candidate will say if they’re asked, in front of the supervisors, whether they feel all right, whether they can handle the pressure and workloads, and whether they’re happy with their developments?”

“When, during my PhD track, I started my specialised training, I noticed that in the clinic, I was evaluated much more broadly, based on multiple competencies. I received an app, with which I could instantly ask for feedback about the various skills you need in the hospital. That felt much more modern than the one-sided assessment I received as a PhD candidate.”

You got in touch with your dean, Frank Miedema. As one of the founders of the Science in Transition movement, he’d long been voicing his criticisms (in Dutch, ed.) on the emphasis on publications and citations in careers of doctors and researchers, as well as on the idea of the scientist as a jack of all trades. Miedema, along with staff advisor Rinze Benedictus, ensured that future professors can now present themselves with a substantive portfolio instead of with a cv, of which their publication list tended to be the most important aspect. Along with Benedictus and other young researchers, you founded Young Science in Transition. Why?
“In my view, the position of the young researcher within Science in Transition tends to be a bit of a blind spot. I felt the same when I read the VSNU position paper about new ways of recognising and evaluating. How can we bring these thoughts into practice, I thought, and where do I fit in, as a young researcher?

You sank your teeth into finding new ways to evaluate PhD candidates…
“Yes, with the support of Frank Miedema, and later his successor Arno Hoes, two other PhD candidates from Young Science in Transition and I went to my own PhD school of Neurology, with a different draft of the assessment form and the performance review. That was well-received, partially because they’d already been searching for ways to put into practice this desire to change the supervision of PhD candidates. Earlier this year, the form was implemented in my PhD school, and now, I’m also receiving a lot of support from the Graduate School of Life Science to introduce that approach in all fourteen PhD schools there.”

What are the most important changes?
“First of all, the question about the number of publications, abstracts, and awards has been replaced. PhD can now indicate what they feel their most important achievements are. They’re allowed to name two. And that can just as well be a speech at a conference, or successful contact with a patient.

“Furthermore, there are now two separate conversations for the performance review: first one with the mentor, and then one with the supervisors. They now use a PhD competence model that lets PhD candidates reflect on the competencies they’ve acquired. The model had already been used at the graduate school occasionally, but many PhD candidates had never heard of it.

‘PhD candidates can now indicate what they feel their most important achievements are’

“The model isn’t just about scientific qualities, but also about teaching, showing leadership, personal development, etcetera. Then, through spinplots, it shows how you feel you score on certain aspects. Based on that, the PhD candidate can discuss their own development with their supervisors.”

I can imagine a change like this doesn’t happen without a struggle?
“No, one of the recurring questions in the conversations with the graduate school was: is there any proof that this model actually helps a PhD candidate? I didn’t have any, but of course I countered with: was the old approach evidence based, then? It wasn’t.

“Some of the PhD candidates were also hesitant to use such a subjective approach, with ‘jibber-jabber’ about competencies. The art is to make such an approach practical and manageable. You shouldn’t burden PhD candidates with a mile-long checklist. Moreover, there’s the risk of making them think they’re supposed to be good at everything. And that’s what we want to get rid of. A PhD track is a learning process that makes someone a competent researcher, but the path can be different for every PhD candidate.”

‘Was the old approach evidence-based, then? It wasn’t’

“We’re already busy evaluating what PhD candidates think about this. So far, it’s looking good. PhD candidates feel as though they’re more properly prepared, and that the conversations are more constructive and focused on content. They get more out of it. I, for instance, could show that I have difficulty dealing with disappointments and hierarchy. That was then the basis for a very educational conversation.”

What do you think is the greatest benefit?
“The most important thing to me is that PhD candidates are much more in charge, now. They have an instrument that helps them think about what they want, and how to do it. That’s important, because not all PhD candidates continue to work in science.

“I know we’re not nearly there yet. Developing a new procedure doesn’t mean you’re changing the culture. There are definitely going to be professors who think this is nonsense, because they’re so used to the fact that bibliometric data are important. They’ve built their own careers on that foundation. And when I become a Postdoc, I’ll probably face some of the same familiar hurdles. It’s probably quite telling that recently, my friends pointed out that I still present myself on LinkedIn with my published scientific articles. A mistake.

‘You shouldn’t burden PhD candidates with mile-long checklists’

“But I’m convinced that we’re slowly changing things. A few years ago, I probably wouldn’t have been given the space to work on something like Young Science in Transition, next to my research and my work in the hospital. No one can ignore this development.”

And what has this whole track brought you?
“It’s cost a year and a half, but I’m incredibly proud that I can now tell this story. The support of my faculty and the PhD school has helped a lot; it has to work both ways. But mostly I’ve had to do a lot by myself, talk a lot, try to convince people.

“Ironically, this track has helped me develop skills that at first glance have nothing to do with my qualities as a scientist, but that’ll help me get further in science. That’s exactly the type of development I want all PhD candidates to have.

“Recently, for instance, in a discussion about the setup of a large-scale Covidstudy in various Dutch hospitals, I managed to garner support from several parties – who a first had some reservations – to collaborate in a major investigation into cerebral infarcts. I thought that was great. That’s when I benefit from the leadership skills that I’ve developed in my battle for this new way of recognising and evaluating.”

Annemijn Algra studied medicine in Leiden. She’s a PhD candidate in Utrecht and a neurologist in training at the UMC Utrecht and the St. Antonius hospital in Nieuwegein. She’s also the leader of the second year of the thinktank Young Science in Transition, a group of young researchers who, through conferences and workgroups, want to contribute to changing science practice. (Photo: Erik Kottier)