‘She sucks at teaching and has no didactic insight’

"You’re a whore," "They should fire you on the spot," "You’re the antichrist," "You’re much too young to assess me." Some of the remarks made by students in course evaluations are downright offensive, and that's a persistent problem. Currently, all courses at Utrecht University are evaluated through the Caracal system. At the end of every block, students receive an e-mail requesting them to fill in a list with tens of questions about the courses they followed. Some of the questions are closed, asking students to rate something with a grade between 1 and 5, while other questions are open, inviting respondents to share their opinions about the course and the teacher. The questionnaire is answered anonymously.

Customer satisfaction
UU's faculties have been wondering more and more if this way of evaluating courses even works. The hurtful remarks made by some of the students are but one of the issues encountered. Another problem is the low participation level: an average of 10 to 15 percent of the students answer the questionnaire. Are these surveys indicative of the quality of a course when so few students fill it in? Does it really help teachers to reflect on their performance?

These problems were already raised in 2018, when DUB published a story, originally written by the Higher Education Press Agency (HOP), about course evaluations (available in Dutch only, Ed.). In 2019, the UU put the topic on its agenda following a survey of 100 teachers conducted by Teaching at Utrecht University (TAUU) in 2019 (also available in Dutch only, Ed.) At the time, evaluations were described as reminiscent of customer satisfaction research, instead of a valuable tool to assess the quality of education. After all, supervisors do not always know how to use these evaluations and students do not know how to evaluate the quality of education anyway, not to mention they can be quite rude. The conclusion was that a good score does not necessarily indicate good education. Five years later, has this issue faded into the background? DUB immersed itself in the matter and spoke with several experts.

Making it personal
In the article published in 2018, DUB already mentioned that teachers were sometimes confronted with swearing students in these evaluations. "Evaluations should not become a culture of judgement," Rector Henk Kummeling said at the time. "If they do, we will have to pull the plug." He acknowledges that things haven't gotten any better since then. "I have the impression that the number of hurtful comments has increased. This can partially be attributed to polarisation but also to social media, where shouting out your unfiltered opinions has been normalised."

The anonymous nature of the questionnaire is the main reason for students being so harsh in their judgments. Respondents tend to evaluate a course negatively when they find it hard or when they get a bad grade, for example. Other factors, such as the teacher's age, gender, language skills, presentation style, and the assignments' difficulty level also play a role. Recently, a study showed that women tend to get more negative assessments than men (available in Dutch only, Ed.). In addition, women are more often judged on their looks.

This doesn't come exactly as a surprise. In 1996, American psychologist Ian Neathy wrote a frequently-quoted article about course evaluations, featuring tips on how to score well with students. Some of his advice: make sure you’re a man, turn your lecture into a performance, give mostly working lectures, don’t dress too provocatively, refrain from expressing your own opinions, and make sure that students get good grades. Ironic recommendations, of course, but there is some truth to it.

Judged on political preferences
Philosophy teacher Floris van der Burg, from University College Utrecht, gives an example of how it works. "We had a teacher who worked hard to teach a difficult course the best way she could. Many of the students obtained little more than a passing grade, something that UCU students aren’t used to, they usually get high grades. So, in the evaluation, they blamed the teacher for their scores. One student even called her the antichrist."

In these polarised times, this seems to have increased even further. Teachers are now judged on their political or sexual preferences as well. That's what happened to a freelance teacher who taught a course at UU’s Honours Class, whose evaluation DUB was allowed to read. "She sucks at teaching and has no didactic insight, she’s constantly transphobic, homophobic, and racist. She uses her lectures to promote her VVD views (VVD is a centre-right political party in the Netherlands, Ed.) even when it is not relevant to the class." Others called her "The worst teacher ever." In contrast, there were just as many positive responses about her. 

The environment philosopher Floris Van den Berg, from UU, says in his recent book Het spook van woke (The Woke Ghost, Ed.) that the way students criticise courses has changed. It used to be aimed at literature, exams, or the time certain classes were offered, whereas, today, students are more likely to complain about being offended, or that the content contains too many authors with a male, Western, perspective, or that students don’t have the freedom to express themselves.

Teresa Pappalardo, who studies Philosophy, Politics and Economics (PPE) acknowledges that students are sensitive to teachers' political opinions. "Some teachers want students to engage in a discussion, inciting them with provocative remarks. Part of the students take these expressions literally and call the teacher a racist, for example. They also express their discontent in their evaluation of the course."

Low response rate
In addition to the offensive remarks, many conversations show that there’s another problem with course evaluations: the low response rate. An estimated 10 to 15 percent of students answer the survey. What can be said about the quality of a course if so few people evaluate it? "Digital course evaluations merely give an impression of how students experience a course," says an article from the Faculty of Humanities, published in 2020. 

Students don’t take the effort to fill in the digital questionnaire with detailed questions for every course. "Usually, at the end of the course, we receive an e-mail asking us to evaluate it. Students have already finished the course by then. We get very little information about why we're supposed to evaluate it and what the university is going to do with that questionnaire. That’s not motivating at all," Psychology student Claire Bruls explains.

Besides, every single course is supposed to be evaluated, which does not encourage students to fill in the questionnaires. I also means that many of the evaluations are of little use – after all, a course that’s been taught often will not be assessed differently all of a sudden.

The larger the setup of the course, the less willing students tend to be to fill in an evaluation. Master's student Marcel explains his reasoning: "I don't feel the need to fill in an evaluation about a big course. But, if it's a small-scale class with a good atmosphere, it makes sense to support the teacher and fill in the evaluation."

Teachers' assessment
A thorn in teachers' side is that evaluations may be taken into account in their assessment. In the past, course evaluations were often included in assessment interviews or required for attaining a basic teaching certificate (Dutch acronym: BKO) and a senior teaching certificate (SKO). Currently, the consensus is that course evaluations should not be used to assess teachers but, at the Faculty of Humanities, evaluations are still used in assessment interviews in certain cases. "But it hasn't been the standard since 2019. When teaching ability might be an issue, course evaluations can be used exceptionally. If it comes to that, the Education Director always plays an intermediary role. The evaluation will be used as the starting point of the conversation. The teacher will be asked how they deal with criticism, for example," explains Humanities policy officer Siebren Teule. Teachers can also submit their own evaluations as part of their BKO portfolio, to show how they use students' feedback to make adjustments to the courses.

However, what may still be an issue is how course evaluations are included in the assessment of external teachers, such as guest teachers ministering honours classes or UU teachers who are invited to teach a course at University College Utrecht. According to UCU teacher Floris van den Burg, course evaluations do play a role in the decision to continue the collaboration. "You won't be fired immediately because of a negative student assessment. But, here at UCU, course evaluations do influence the decision of whether or not to allow a certain teacher to keep on teaching a course. I have also seen management asking a teacher to adjust a course because students did not agree with the assignments they were presented with," says van der Burg.

Who can access course evaluations? 
Last but not least, there’s a discussion about who has access to the data collected through course evaluation surveys. As a general rule, the data is accessible to teachers, education coordinators, educational directors, and course committees. The latter can be given advice for next time, based on the evaluations. However, teachers are also expected to reflect on the survey's outcomes and where the advice is applicable.

At some faculties, teachers and course committees post their conclusions on Blackboard, so that the students who have taken the course can read them. In some cases, students interested in taking the course can read them too.  

Teachers can get disturbed or embarrassed when course evaluations contain remarks about them that aren't relevant for the course, especially when the remarks are visible to others. For this reason, the Faculty of Science is giving teachers the option to have distressing comments removed from the evaluation. This is done by a dedicated committee. 

New choices
Efforts to improve the situation have been underway since 2019. Each faculty has contributed a memo on a new approach and the university as a whole has prepared a policy note. "Following the decision to use Caracal for all courses, we've received signals from the faculties that there was a need for more policy choices and more flexibility in the implementation of course evaluations," says Renée Filius, Head of the Education Department. "In the policy note, we indicate how to use course evaluations and how to take objections into account, but, ultimately, it’s up to the faculties to make the choices. It's also a matter of customisation."

Good quality management
"The evaluations are part of a healthy culture of quality, in which we ask ourselves if we have our stuff in order, how courses are going, and whether we are headed towards the right direction," declares UU Rector Henk Kummeling in HOP's article. Now, the rector says: "I still think that appropriate quality management is necessary, but I have started to wonder whether the classic setup we're currently using, with anonymous questionnaires, is the right one. Shouldn’t we experiment with other forms of evaluation, which would be more integrated in teaching and in which teachers and students discuss things together?"

Marlies van Beek, educational advisor and trainer in the Department of Educational Advice & Training at the Faculty of Social Sciences, also considers it necessary to maintain course evaluations. "We need to know how students experience their courses and curriculum, and where things can be improved. Average scores don't say much, regardless of whether the response rate is high or low. However, you can distil useful tips thanks to more or less standard questions, allowing students to explain their answers or provide additional comments. But it’s certainly worth thinking about how evaluations can be improved to address these issues."

About this article
This is the first part of an article about course evaluations at Utrecht University, which was made possible by the Foundation for the Stimulation of Educational Journalism. Part two will be published in January 2024.