Examinations and high stakes decision making in the era of COVID-19

For many of us, our progress through school was punctuated by a series of high-stakes examinations that determined whether we would move to the next grade level, graduate from secondary school, or enter university. In my case, it was the Leaving Certificate examinations in Ireland which determined whether I would graduate from secondary school and which university course I could take. It never occurred to me at the time that there might be other, equally valid or better, ways of making these high-stakes decisions about my future. 

Over the last few months, however, as coronavirus (COVID-19) pandemic forced the closure of over 90 percent of the world’s schools and required the imposition of strict social distancing measures, ministries of education everywhere had to rethink how to make these kinds of high stakes decisions about their students’ futures, including the extent to which examinations could be retained as part of the decision-making process. 

Where did countries come out? A recent survey of 118 countries, carried out by UNESCO, UNICEF, and the World Bank to understand how countries’ education systems were responding to the crisis, found that the most common responses by countries included: (i) shifting to online administration of their high-stakes examinations; (ii) postponing the examinations to a later date; (iii) continuing the examinations as planned, albeit with various social distancing/safety measures; or (iv) canceling the examinations entirely and using alternative sources of information to make decisions about students. 

In this blog, I want to focus on the last approach – i.e., canceling – since this is, in many ways, the most extreme response, and raises interesting questions about the extent to which other sources of information can be used to make these kinds of high-stakes decisions. The United Kingdom (UK) is a good example. On March 18, 2020, the UK Secretary of State for Education announced that the summer 2020 A-Level exams (used to certify school leaving and for university entrance) would be cancelled in order to help fight the spread of COVID-19. Instead, each student was to receive a “calculated grade” for each of their subjects, to be determined and standardized using five sources of information. 

1. Centre Assessment Grades (CAGs) – predicted grades for students for each of their subjects provided by their school;
2. Subject-wise ranking of students within each of the CAG levels provided by their school;
3. Expected grade distribution for that subject at the national level;
4. The school’s historical grade distribution for that subject over the past one to three years; and
5. The school’s predicted grade distribution for that subject based on results on other examinations (e.g., the General Certificate of Secondary Education results)

CAGs (the most important piece of information out of the five) are supposed to be decided by the subject team in a student’s school and signed off by the Headteacher before being submitted for standardization. The information used to calculate a student’s CAG may include (among other things): classwork, homework, any non-formal assessment carried out by the school; AS Levels (similar to the first year of an A Level course) performance if the student sat those exams, and mock (practice) exams. 

Not surprisingly, this “calculated grades” approach has both supporters and opponents. Students who support the plan feel that it will be a better reflection of the work they’ve put in over the years. Those against it point out that the ‘rules of the game’ are being changed at the last minute and they did not expect that assignments and other work they previously viewed as low-stakes or formative in nature would be used for high-stakes decision making. 

There are also those who point to technical issues with an approach that relies so much on distributions and predictions. For example, students attending smaller schools may be at a disadvantage because their school’s predicted grade distribution may be less stable due to less data. There also are concerns that the calculated grades for students from disadvantaged backgrounds may suffer given that teachers tend to underestimate the abilities of students from such backgrounds. 

My home country of Ireland has also decided to cancel the Leaving Certificate examination and instead issue calculated grades to students. The formula seems less complicated than the UK’s, but the issues are similar. A key issue in both countries is the role of teachers in the calculated grades process. While in some countries (e.g., Australia), teachers are used to (and welcome) playing a key role in the determination of their students’ grades as an input to high school graduation or university entrance, in others, teachers want as little as possible to do with the process. In Ireland, for example, there are concerns that teachers will face pressure from parents and students to provide certain grades and may face recriminations if they do not. There also is a belief in the inherent value of an external ‘objective’ measure of students’ performance as they transition from compulsory schooling. Finally, there is a sense that teachers have not been trained or given enough time to feel confident in their ability to make these kinds of high stakes decisions about their students’ futures. COVID-19 has made the whole process feel a little rushed.  

At this stage, many of the above concerns and scenarios have yet to play out since students have yet to be issued their final calculated grades. In the meantime, it will be important for the UK, Ireland, and other countries using such alternatives to their traditional examinations to collect as much evidence as possible on the process, both to support better high-stakes decision-making using these alternate approaches as well as to facilitate evaluation of these approaches as options for the future. After all, we owe it to students to use the best evidence possible – examination or non-examination related – to make these kinds of high-stakes decisions about their futures. The circumstances produced by COVID-19 may well yield some useful insights that will help countries to build back infinitely better in this area.