Alternative approaches to high-stakes examinations during COVID-19: Lessons from Ireland

Among the many changes wrought by COVID-19, one that has been especially disruptive for millions of students worldwide is the cancellations or modifications of high-stakes examinations. In many countries, examination results determine whether students graduate from school, access the next education level, or enter the workplace. Around the globe, millions of students have been affected by the changes made to examination systems during the pandemic. Over the past year, I’ve been getting a close-up view of what it’s like to prepare for a high-stakes examination during COVID-19 by watching my nephew study to take the 2021 Leaving Certificate Examination (LCE) in Ireland. Ireland is a good example of how the alternatives that countries have used during COVID-19 may provide useful lessons in how to improve examination systems moving forward.

The alternatives

In early 2020, with COVID-19 resulting in school lockdowns and social distancing, Ireland’s response was to postpone its traditional LCE (used to certify completion of secondary schooling and determine entry to higher education) and replace it with “calculated grades.” To create the calculated grades, teachers were asked to rank their students in each subject area and assign them a percentage mark; these marks were then sent to be standardized by the Irish Department of Education (DoE). In 2021, with the pandemic under better control but final year students having experienced two periods of suspension of in-person teaching, it was decided that students eligible to take the LCE would be given three options for each subject – they could either (i) opt for an “accredited grade” (an evolution of calculated grades), (ii) take the LCE, or (iii) do both. If they chose both, they would be awarded the higher of their accredited grade or LCE grade.

What are we learning from such approaches?
 

1. Change is possible, but it requires trust

Due to the historic view of the LCE as an impartial and external measure, many Irish teachers found themselves in an unprecedented position when calculated grades were introduced in May 2020. Nonetheless, they went on to implement the DoE’s grading procedures and generally felt confident about the marks they submitted in June 2020. However, the decision, in September 2020, to allow students to see their individual ranking data created strains with the teachers who had created these rankings and raised questions about the use of accredited grades in 2021.

While the plan is to only use the traditional LCE in 2022, the experience with calculated and accredited grades showed that Irish teachers were able to make judgments about their students as an input to high-stakes decision making. At the same time, the experience highlighted that any effort to introduce teacher judgment as a permanent part of the examinations process would require a very solid foundation of trust between all the parties.
 

2. Giving students options can help them perform better

The three options provided to students in 2021 proved popular, with the vast majority opting for some mix of accredited grades and/or written examinations, depending on the subject area. Just 8 percent opted to only accept their teachers’ assessed grades and did not register for any written exams. In cases where students had both a written examination grade and an accredited grade, they were automatically awarded the higher of the two. It turned out that in just over half the cases (52.5 percent), students ended up being credited with their accredited grade. The accredited grade and examination results were the same in 31 percent of cases, and in a small number of cases (16 percent), students achieved better results on their written examination.

There is still a lot to unpack here, but one takeaway might be that providing assessment options gives students the chance to show more effectively what they know and can do. Many examination systems are built around a single approach to measuring student knowledge and skills. What if we gave students more options to show what they know and can do?
 

3. Better results do not always lead to better opportunities…

Having more choice helped students perform better, and the 2021 Leaving Certificate results were 2.6% higher overall than the 2020 results. However, this increase also pushed up the points thresholds to get into the most popular higher education courses. Some students who achieved perfect scores still did not get into their chosen course because so many others also achieved perfect scores and places were limited.

Thus, another lesson might be that if too many people do well on the measure (in this case, Leaving Certificate results) used to decide access to limited opportunities, the measure’s discriminating power, and hence its value for that purpose, will diminish. This is where systems that rely on multiple sources of information to make decisions about student entry into higher education can have an advantage.
 

4. …Unless you change the system

The high results and the knock-on effect on the points required for certain courses was less of an issue in Ireland than it might have been in other countries since Ireland has a highly differentiated higher education system that has “something for everybody.” (Hence, those who did not get their first-choice course still had other good options.) The equity issues were mainly about decreased access to the most in-demand courses for otherwise qualified students. Trying to mitigate this, the Irish government increased the number of available places in high-demand courses.

However, this still leaves the question of whether it is fair to keep increasing the number of points required for a particular course. If a certain level of knowledge is required to succeed, should the bar not simply be set there? If there are more qualified applicants than places, and places cannot be increased, could it be fairer – as some have suggested – to randomly allocate spaces to qualified students rather than to assign them to students with the marginally higher scores?

Ireland is not unique in having to deal with examination disruptions due to COVID-19 and these kinds of scenarios have been playing out in many other education systems over the last two years. As the Irish experience shows, however, the alternatives being pursued by countries have sometimes pushed the envelope in ways not formerly seen. These experiences can provide innovative insights that might eventually produce better equity and quality in examination systems moving forward. To derive the benefits, it will be important to carefully document and reflect on these diverse experiences.

* This blog benefited from substantive inputs by Diego Luna-Bazaldua, Julia Liberman, and Victoria Levin (World Bank); and Michael O’Leary and Vasiliki Pitsia (Center for Assessment Research, Policy and Practice in Education).