'RIVM research about students' mental health not representative'

Titled ‘Monitor of mental health and substance use among higher education students’, the he long-awaited survey (document in Dutch) shows that two out of three students felt emotionally drained during the lockdown of the third Covid-19 wave. A quarter of respondents stated they were sometimes tired of life, wanting to go to sleep and never wake up. They also reporting drinking lots of alcohol and smoking cannabis a lot.

The results shocked student organisations and higher education institutions across the country. The Minister of Education, Ingrid van Engelshoven, used the report to stress that decreeing a new lockdown for universities would pose huge risks for the mental health of our students.

But are students really that depressed? Han van der Maas, Professor and Head of Psychological Methods at the University of Amsterdam, suspects not. “The survey that is scaring people so much is not valid”, wrote the professor in the Dutch newspaper De Volkskrant (text in Dutch). “Only 11.7 percent of the 250,000 students approached actually completed the questionnaires. Students with mental issues are probably much more likely to respond.”

Weak spot
“In recent years, I have seen a lot of surveys that, just like this one, have been conducted well in many respects, but they have one very weak spot, which is the way the sample was composed. People are invited through a website or post to complete a questionnaire, but the response rate is often so low that you don’t know what kind of self-selection took place,” said the professor.

In his view, the RIVM report is an example of such a survey. “It most likely leads to a substantial distortion of the results. The roughly 90 percent of students that did not complete the 20-minute questionnaire would probably have responded differently to questions about loneliness and being tired of life.”

Van der Maas refers to international studies on happiness. “In 2019, the Netherlands was once again in the global top 5, with a score of 7.5. We are now in the grip of Covid-19, which doesn’t make life any more enjoyable. But the score of 6 that students gave in the RIVM survey – which uses the same survey method – is the same level as Mexico. People there are living in very different conditions from our students.”

Cautious
Health psychologist Peter van der Velden, senior researcher at Centerdata and former Professor of Victimology at Tilburg University, shares Van der Maas’s critical view. “The high level of publicity surrounding the survey is not justified by its quality. It’s perfectly possible to get such a low response but the researchers then have to be very cautious with the results. Especially if you don’t take into account how things were going with those students before Covid-19. Then you don’t actually know whether the problems have increased or decreased.”

Van der Velden and his colleagues are studying wellbeing and Covid-19. “We’re doing that by using the LISS panel, which is based on a random sample of the Dutch population. The panel has more than 6,500 members. What we saw in December 2020 is that young people have more mental health issues than elderly people, but that was also the case in the year prior to Covid-19. So yes, young people have more mental health issues than elderly people, but that isn’t solely because of Covid-19.”

In his view, the five-yearly survey among young people, which his institute conducted once again in December, shows a slight increase in mental health issues. “But nothing is happening that is of immediate or major concern. The same emerges from the annual survey by Statistics Netherlands, which is representative as well. Nothing indicates that things are going horribly wrong with youngsters of that age in the Netherlands.”

UvA professor Van der Maas would be interested in conducting follow-up research with the questionnaire of the RIVM survey. “I would be inclined to conduct some of the survey in the old-fashioned manner – on the campus of an institution, among around a thousand students. It’s a lot of work but the non-response would be a lot lower. If the same numbers were to emerge, I would have to admit that I was wrong, but I have the feeling that you would really get different data from it. With a response rate of 75 percent you would be able to check the present survey result properly.”

Too easy
In a written reply the Monitor's authors admit that the low response rate is a weak point in their research. “We describe this at various points in the report and have (…) been cautious in drawing our conclusions.”

The researchers do not rule out the possibility that a relatively large number of students with mental health issues completed the questionnaire. “If a bias of that kind arose, it might have contributed to a more negative image of students’ mental health than is actually the case in the general student population.”

They say that it is hard to assess how great that possible distortion is. “However, in our view it is too easy – and not correct – to ascribe the findings solely to that possibility.”

To underline that, they refer to another survey by the RIVM and Statistics Netherlands showing that during the third wave of COVID-19 young adults “suffered considerably more from mental health issues than the general population”.

Finally, they stress that almost 30 thousand students from 15 different institutions took part in the survey. “So it isn’t a fact – as has been stated – that ‘few students’ completed the questionnaire.”