NVAO inspector unsure whether programmes have a firm grasp on AI

Higher education programmes in the Netherlands must be accredited to issue a recognised diploma. The accreditation process is regulated by NVAO, which sends experts to educational institutions to assess their facilities, plans, and curriculum.

NVAO has analysed reports written by experts following such visits, focusing on a specific topic: generative artificial intelligence. How are degree programmes dealing with AI? What do they teach students about it? Are there clear rules regarding the use of generative AI tools? Do teachers understand what AI can do? 

Struggling
NVAO notes in a “Preliminary Exploration” that interest in generative AI has risen sharply. ChatGPT was launched at the end of 2022 and proved to be astonishingly good at generating text. Since then, degree programmes have been struggling with the consequences.

Or are they? A closer analysis of 194 recent assessment reports has revealed that research university programmes are paying more attention to AI and its consequences than programmes offered by universities of applied sciences. NVAO has not provided further explanation for this observation. It should be noted, however, that the situation may have improved since the reports were written, as quite some time passes between the experts' visit and NVAO's final approval.

In any case, NVAO observes that programmes have been focusing mainly on the consequences of generative AI for students' assessment, while paying much less attention to AI's influence on students' future careers. What skills should students learn if AI is soon to be a standard part of their work?

Graduation quality
NVAO considers it "striking" that the reports discuss assessment in general, but do not discuss the assessment of graduation theses. Both the programmes and the visiting experts do not "explicitly" consider the risk of AI use for writing final theses. "The inspection reports are not clear whether programmes and their examination boards have a handle on this situation and whether they have discussed it with the panel."

NVAO does not draw any further conclusions. Its chair, Arnold Jonk, writes: "The frequent calls for frameworks and certainty are logical, but we will probably have to live with uncertainty for some time."

The exploratory report also states that "panels are not obliged to engage in discussions about the use of generative AI and are not required to report on this." In the meantime, the inspector is considering whether quality assurance procedures should focus more on generative AI and, if so, what the expert panels need to make a proper assessment.

AI literacy
Last year, the Education Inspectorate also raised the alarm about generative AI and graduation theses, having identified a "major risk" to the quality of graduation. Examination boards find it difficult to detect or prove AI fraud, and they are not given enough time to learn about its risks.

NVAO agrees with this conclusion, which is why it is asking educational institutions to improve teachers' AI literacy and think more carefully about the role of AI in their teaching.