Testing students when they're at home? How does that work?

‘How do I know that a student has taken the test alone?' and 'If it turns out that the test has been made abnormally well, can I make the norms stricter?’ Didactic advisor Melvin van Puffelen saw quite some tension among the Utrecht teachers when they were told that not only their teaching, but also their testing of block 3 had to take place online.

This spring, Van Puffelen will be hosting a number of webinars from Educate-It where teachers can ask questions about online testing. During the past four meetings, 120 teachers were present from out of their homes. "Initially, we received a lot of questions about how instructors can keep an eye on their students. In my opinion, we were able to help dispel most of the unrest about that aspect".

‘You soon saw that teachers came up with creative solutions themselves’

Nearly 1700 courses must be completed with an exam in the second semester. Because of the corona crisis, this is not always possible in the way that was agreed upon. When it comes to a paper or essay, there are not a lot of issues. They can simply be handed in. But this is more difficult, for example, for the now familiar digital tests that have to be taken using Remindo software in an examination room of the Educatorium. These kinds of exams cannot simply be done at home by students. If only because Remindo does not have a plagiarism check with which answers can be compared.

But Van Puffelen slowly saw some relaxation, also because the university board insisted that lecturers would not aim at maintaining one specific testing moment for a certain course. They could also consider other forms of testing. Moreover, exams or test assignments that were impossible to take online could possibly be postponed. This mainly involved practical work, fieldwork or, for example, a lawyer's advocacy course.

Together with specialists from Education Consultancy & Professional Development, Van Puffelen provides tips to approach testing differently. These may include technical tools to let students create and hand in videos or pitches, for example. However, instructors were also given advice on introducing multiple midterms or open-book exams that must be taken within a certain period of time in order to prevent secret collaboration. "You soon saw that teachers came up with creative solutions themselves. For example, someone instructed students to make a summary of a book. That is of course possible as well, if it matches the learning objectives".

‘The important thing is that we have enough to hold on to’

Students of Educational Sciences can mainly expect open-book exams these weeks, says education director Jeroen Janssen. These are held for six bachelor's courses at times when they had already been scheduled. Students submit their answers afterwards via the electronic learning environment Blackboard. That system does offer the possibility to check whether students have given answers that are too similar. Janssen laughs: "Or if they make the same strange language mistakes."

Janssen acknowledges that the method is not fool proof, and he has also heard that the whatsapp strategies are already shared among students. "We cannot prevent this, but given the current circumstances it is not our task to make the perfect test. The important thing is that we have enough to hold on to."

He emphasises that the exam is never the only thing that determines students' grades. There is always an assignment or a paper to work on. He also expects that the greater emphasis on questions of insight instead of knowledge and facts will make it difficult to consult with each other. "Furthermore, we have told our students honestly that we know that you can cheat, but that we want to show our trust. This has always been the approach of the university board. I think students understand that, although of course you don't know what happens when stress strikes during an exam."

There are many open-book exams scheduled for Law students as well these weeks, according to education director Bald de Vries. "We had to adjust the form and approach a lot to ensure that students hand in their own work. There are many more questions of insight in which critical reflection is particularly important. In addition, in some cases we have ensured that not all students receive the same questions or see the questions in a different order".

De Vries also knows that it is impossible to rule out that students may still consult during a test. "This is an emergency situation. We do as much as we can to prevent it. Furthermore, students feel that they will eventually lose out if they give in to temptation as well. I do have a good feeling about that."

Before Janssen and De Vries were allowed to offer other tests at all, they had to have the approval of their board of examiners. In Educational Sciences, this went quite smoothly. Janssen: "The most important point of discussion was not so much the way of testing, but the graduation research in which a number of bachelor's and master's students have encountered difficulties. Collecting data at schools, for example, is now a difficult task".

The chairman of the board of examiners of Law Emanuel Van Dongen speaks of a "fast and intensive" process that ultimately went without a lot of problems. "I am in contact with education director Bald de Vries every day, and we discuss the choices made in the various courses. This way I can quickly make adjustments, if necessary".

Van Dongen saw it mainly as his task to check whether teachers were keeping an eye on the learning objectives and goals during the assessment. But the potential fraud problem had his attention as well. "The course coordinators were given guidelines on how to prevent plagiarism. Questions of insight are already throwing up a barrier. You can also hold an oral exam, although that's difficult with our number of students."

It can't be perfect under the current circumstances, thinks Van Dongen. "For the time being, it is a matter of finding a balance between pragmatism and guaranteeing quality. We think we've found it now."

‘We get a lot of questions about online surveillance’

But what about the now much-discussed 'proctoring'? The Faculty of Medicine is using this surveillance mechanism for regular tests in Utrecht this block for the first time. Several UU programmes have already gained experience with it during selection procedures. But as of this week, a few hundred Medicine students of some courses will have to identify themselves at home via a webcam and will be observed. They will also have to share their screen and are not allowed to talk or pause the test.

Educate is currently researching whether the supplier of the software has sufficient capacity to enable online proctoring for a limited part of the Utrecht exams of period 4. According to Melvin van Puffelen, Educate-It is also looking at less far-reaching methods in which, for example, students are only asked to turn on their camera so that they can be observed by teachers or examiners. This could be an option for small groups of students. "We get a remarkable amount of questions about all those kinds of means."

Jeroen Janssen has his doubts whether proctoring should be put to use extensively. He wonders what happens to the collected images and data. Bald de Vries does not yet see how eight hundred law students can be monitored at the same time. But above all, in his opinion, the time is not yet right for camera surveillance. "I don't think it will contribute to a better state of mind for our students if we suddenly introduce this now. Students feel pretty insecure anyway. We have to be careful with that."

'Cameras don't contribute to a better state of mind for our students at the moment'

The situation of their students is something the two directors are worried about anyway. Not to mention the psychological and physical damage caused by the corona crisis itself. What if students can't find the peace and quiet to study in their student room, let alone take an exam? Or maybe there are students without a good internet connection? The directors have committed themselves to keeping an eye on the situation and to evaluate not only the teaching but also the exams properly and quickly. They hope to be able to find improvements for the next block as well. For example, De Vries is considering panel discussions with students. Janssen also points to the many part-time students, some with a job and family, and says: "We cannot blame students for this situation. We will have to be accommodating and offer customisation where necessary".

However, neither of them wants to settle for the idea of setting flexible norms in advance. De Vries: "Everyone is aware of what is going on. But let's just await the analyses of the exams. And then - just as there has always been - there is the possibility of adjusting the grades".

‘Exams must be fair for students, but also manageable for teachers'

In the meantime, there is always the question of whether the teachers will be able to keep it all up. More questions of insight, for example, quickly means more work to grade exams. The university board has already indicated that teachers are not bound by the review period of a maximum of ten working days during the crisis. Janssen: "We reminded the course coordinators that exams must be fair for students, but above all they must be manageable for teachers. Even if that means asking only six instead of ten questions."

De Vries fears the moment when online lecturing and testing will become "the new normal". "Then, all the positive energy of having to change will be gone, but the extra work will remain. We have to watch out for that."