Six reasons why it’s not standard practice at UU
Analysis: the long road towards anonymised examination
Photo: Shutterstock, DUB
Careful optimism. That's what some members of the council at the Faculty of Humanities feel following a meeting on November 4. They would like for teachers to start using anonymised examinations as much as possible and the faculty board announced a "feasibility study" into the practice. Will they finally succeed in making unbiased examinations part of UU's policy?
This is the latest chapter in a years-long discussion that is not restricted to the Humanities department. To understand what makes this topic such a hot potato, one needs to go back in time.
Biases
Sander Werkhoven and Ingrid Robeyns got the ball rolling in 2018. The two Ethics teachers from the Faculty of Humanities started an experiment about anonymised examinations in which, for a couple of months, students did not fill in their names on their examination forms. Instead, they only provided their student numbers. This way, teachers did not know whose exam they were grading. This may sound like a simple solution but, according to the initiators, its implications were of great importance.
At the time, Werkhoven and Robeyns wrote an article (in Dutch only, Ed.) for DUB about why anonymised examinations should be the standard at UU. In their view, this form of grading is essential for the mitigation of biases and prejudices that may play a role in a teacher's grading.
Did the student speak up during lectures? Do they look nice? Did they get good grades in previous exams? Do they have charisma? These are all things that may make a teacher more or less generous in their grading, be it consciously or unconsciously. The opposite is also true: teachers may be more critical of students from particular ethnic groups or genders. A meta-analysis of twenty studies conducted in 2016 reveals that these kinds of biases do play a significant role in grading. Earlier research shows that anonymised examinations can mitigate this.
That doesn’t mean that all teachers grade their exams unfairly, of course. But it does mean that there is a risk that this might happen. To Werkhoven and Robeyns, such risk should be taken seriously, which is why anonymised exams seem like a simple and efficient solution.
This has been the standard procedure at several universities around the world for years. In the UK, the National Union of Students even held a campaign titled Mark my words, not my name, which influenced many institutions to adopt anonymised exams.
No research possible
Even so, not much has changed at UU since 2018. The ball that started rolling then has been stopped several times in the past few years. The experiment at the Faculty of Humanities yielded positive results, so much so that teachers recommended grading to be anonymised for future exams. Some even said that anonymisation caused them to grade exams more objectively.
Despite the results, the faculty board decided to only encourage anonymised examination, without making it a standard. And that's where things still stand. It was only two years later, in 2020, that the subject was discussed by the university council for the first time.
Two council members, representing the party De Vrije Student, wrote a statement in which they referred to the successful experiment conducted at Humanities. They wondered why anonymised examination wasn't part of UU's policy considering all the scientific evidence pointing towards a serious risk of bias and the fact that the pilot at Humanities had been successful.
UU's Rector Henk Kummeling promised he would encourage faculties to use anonymised exams more often but he thinks that making them mandatory would be a bridge too far. In his view, there are still too many things that are unclear.
His proposal was to first examine whether anonymised examinations are a good fit for UU as something "might be lost" because of it. Kummeling was referring to the personal contact between student and teacher. However, the Executive Board concluded that it wouldn't be possible to conduct such an evaluation in 2020 as most UU employees were busy with all the problems brought about by the pandemic.
Today, Covid plays a much less prominent role at the university but the study still hasn't commenced. At the beginning of 2021, the Executive Board stated that a university-wide pilot to try out anonymised examination was not going to happen because that would be too time-consuming.
Moreover, the new privacy law (best known by its acronym in Dutch, AVG) would make such a study impracticable. According to the AVG, data on students' gender and ethnicity are no longer allowed to be collected, which makes it impossible to study whether some groups benefit more from anonymised examinations than others. The subject has not been discussed in the university council since.
A new push
The ball had come to a halt but the Faculty of Humanities faculty have just set it back in motion. During the council meeting on November 4, they discussed a statement (only accessible to those with a Solis ID, Ed.), written by council members Sarkia Hubers, Myrte Spaargaren and Siebren Teule.
Hubers, together with Lieve Brogtrop, had already written an open letter to the Executive Board a few months earlier, in which they urged anonymised examinations to be recorded in the OER (the education and examination regulation). In the letter, they criticise Kummeling’s argument that a university-wide study is necessary to show the value of anonymised examinations. After all, a number of studies already indicate the role that biases play in comparable institutions. Last but not least, Brogtrop and Hubers argue that the Executive Board had been too hasty in proclaiming the study impossible because of AVG. They say no mind had been paid to alternative ways of researching.
In the statement by the three council members, they once again asked to make anonymised examinations the norm for grading open questions at UU. But they didn't ask the same for other types of examinations such as papers. In their view, anonymisation would be bad in this case as the student’s process is allowed to be considered as part of the grade.
According to these council members, a new standard would help fortify teachers in discussions about the fairness of their grading, not to mention it would raise awareness about the existence of biases.
The Humanities faculty board reacted to the statement the same way all other boards did: with an appreciation for the initiative and the recognition of its importance. Even so, they do identify a bunch of obstacles, such as the possibility of increasing the workload of teachers and other personnel. That's why a feasibility study should be conducted before adopting anonymised exams as the norm.
What such a study would entail was still unclear at the end of the faculty meeting. Council member Teule asked when exactly something could be considered "feasible" but the faculty board could not answer that question. However, there was some optimism in the air too as both the board and the council were willing to work together on this.
Six reasons
The lack of any news in the past few years does make many people doubtful. When asked, Sander Werkhoven, who has recommitted himself to the subject, says that he has little trust in the board's commitment.
What are the obstacles to anonymised examination, an idea first raised almost five years ago? DUB lists six reasons that stand in the way of it becoming part of the university's policy.
1. Board members don't want to look mistrustful of teachers
Obliging everyone to adhere to anonymised exams could be perceived as a sign of distrust towards teachers, as it would imply that the university assumes that some teachers are unable to grade objectively. The faculty board at Humanities, for instance, states that a couple of teachers were offended by the suggestion that they were not objective enough.
That is a tough point: to a certain degree, an obligation would indeed imply a form of distrust but that distrust does have a scientific foundation. Hubers, Teule and Spaargaren write in their statement that it is possible that biases might play a part at the Faculty of Humanities, based on research conducted in comparable institutions. Even if biases would only play a small part, the council members still think they would have a big effect on individual students.
According to the council members, anonymised examinations should mainly be seen as one of the “grading tools that increase the transparency and trustworthiness of a grade, in addition to giving students the possibility to look into their grade, using grading rubrics and giving feed forward”.
2. A student's attitude towards work should be considered when grading them
When anonymised examinations were discussed by the University council, Rector Henk Kummeling mentioned the advantages of knowing which student you are grading. In case of doubt between a pass and a fail grade, Kummeling thinks a teacher should incorporate the student’s involvement in lectures in their grading. This is similar to the opinion of an Art History teacher quoted by DUB in a panel about this subject. She wrote that knowing that a student has made great strides sometimes leads her to give them the benefit of the doubt when grading.
That is a problematic way to view grading, according to Werkhoven. Examinations aim to assess a student's level of knowledge and insight. When a student’s work attitude is part of the grade, two identical exams could in theory receive a different grade. “Would we really want a naturally talented student to work harder for a good grade on a knowledge test if he has progressed less than others?”, Werkhoven wonders.
According to Werkhoven, exceptions can be made for exams where the learning process was an explicit part of the grade. An exam can be graded non-anonymously every once in a while — for example, when a teacher can give a good reason for it. He just thinks it shouldn’t be standard practice.
3. Anonymity isn't always possible
There is a group of students whose anonymity cannot be guaranteed because they have special needs. Dyslexic students, for instance, have to notify the grader so that their situation is taken into account. In small-size student bodies, a teacher might easily find out who a certain exam belongs to.
To Werkhoven, it's a fallacy to say that it's problematic that anonymised exams are not possible for everyone. “If you are not always able to act fairly, that doesn’t mean you shouldn’t ever do it.” Besides, Werkhoven thinks that, if anonymised exams were the standard, teachers would become more aware of their biases in these exceptional cases.
4. Anonymous examinations generate more work
According to a 2020 study by the Ministry of Internal Affairs, two-thirds of the people working in higher education institutions have a high or very high workload. It is, therefore, not surprising that the board of the Faculty of Humanities is worried about the increased work pressure that anonymised examinations may bring.
An obligation to adopt anonymised exams would require adjustments. Teachers would need to use new examination papers for in-person exams, for example. But this paper already exists and most exams already provide the option to anonymise the grading process.
Additionally, one needs to be careful when entering grades manually in an anonymised system: it's easier to give a grade to the wrong student number than to the wrong student name. According to Huber, Teule and Spaargaren, the most important workload increase, which they still describe as “light”, would be caused by the adjustments that need to be made. But this extra work would occur only once.
According to the three council members, when weighed against the advantages of anonymised examinations, the workload increase is worth it.
5. Biases are hard to demonstrate
As previously said, the Executive Board proposed a study into grading biases before the pandemic started in 2020. Eva Klaver, one of the UU Council members that wrote the statement about anonymised examinations, was satisfied with the proposal. If that study indicated that biases also exist at UU, the board would be forced to incorporate anonymised examinations into its policy.
But that study has never been conducted. The Executive Board says a pilot would have been too complicated and time-consuming. After all, determining whether bias occurs at UU would require a researcher to first verify which students bear the consequences of such biases. This researcher would have to look into gender, ethnicity, social class, appearance and earlier grades — something that would have been challenging even without the AVG. Then, they would have to make a comparison. How do the results of students who run the risk of a biased grade differ when they are graded anonymously? It is hard to find the best course of action.
It is also true that such a study would be time-consuming. It is doubtful whether the role of biases at UU could ever be proven. On the other hand, Saskia Hubers and Lieve Brogtrop do have a point about earlier studies. Although they don’t prove that biases do occur at UU, they do prove that the risk of this happening exists. The question is whether this is reason enough to establish anonymised examinations as a centralised UU policy.
6. Students don't care much about anonymised exams
According to Sander Werkhoven, the main reason why anonymised exams are not a university-wide policy is a lack of interest from students themselves. The Mark my words, not my name campaign from the UK never made its way across the Channel. The debate has been restricted to faculty board meetings, for the most part, always initiated by idealistic teachers, in Werkhoven's view. In all likelihood, they are unable to impose change. It is important that students demand to be graded anonymously too.
Among the student members of the university council, the change already seems to be happening. Ismaïl Sarti says he has been following the subject for a while. He is delighted to hear the news about the Faculty of Humanities. Last Monday, during a meeting with the Executive Board, he raised the topic for the first time.
He sees what Werkhoven means when he says students aren't interested enough but notes that the subject already matters to some of them. It was even one of the reasons why Sarti ran to become a student member of the university council: he thinks that students should become more aware of anonymised examinations and is planning to use social media to engage them.
Ismaïl is confident that something can change this time. When asked about how he will do this, he answers simply: “Being determined and tough.” In his view, if a time-consuming study is necessary, then that study should be performed regardless. He acknowledges the high work pressure this would mean for teachers, though: “But work pressure will always be a part of life. That is no reason to ignore important developments such as this.”