Moving high-stakes exams online: Five points to consider

As social distancing measures have been put in place to prevent the spread of coronavirus (COVID-19), governments, universities, and testing organizations around the world have had to decide whether or not to administer high-stakes examinations. These exams are commonly used to make high-stakes decisions, such as selecting students for the next education level or certifying students’ knowledge and skills as they enter the workforce; thus, the results of exams directly affect students’ lives and futures. When properly designed and administered, examinations can enhance equity by opening up access to educational opportunities for students from diverse backgrounds. Likewise, high-stakes examinations can also increase transparency in the student certification or selection processes. 

Prior to the COVID-19 pandemic, high-stakes exams tended to be administered in person, usually in paper format, with students and proctors physically present together in a school or a testing center. In the current context, governments and testing organizations have had to evaluate the extent to which such in-person exam administration is feasible, taking into account health and safety considerations. As investments in the digital delivery of learning content increased significantly over the last several months, so too has the focus on administering high-stakes examinations online to allow students to take such exams from their homes without risking exposure to COVID-19. 

Some testing organizations in the United States have introduced home-based online versions of exams for university admission, course credit or certification purposes, such as the Graduate Records Examination (GRE), the Advanced Placement (AP) exams, and the Test of English as a Foreign Language (TOEFL) exam. Before COVID-19, these exams were administered in person at school or at testing centers following high test security protocols to prevent malpractice. Similarly, some states in the United States, like California, have decided to move professional certification exams to an online format. Likewise, as a precautionary measure against COVID-19, Saudi Arabia implemented its Standard Achievement Admission Test (SAAT) online for the first time using Artificial Intelligence (AI) technology. Some universities in Europe have also implemented final examinations in an online format. For instance, the Imperial College London administered a timed, open-book exam online to its graduating undergraduate medical school students. The University of Amsterdam implemented final examinations online as it continues online teaching for the foreseeable future. 

To administer an online at-home exam, a number of factors need to be in place to ensure that the exam produces valid, reliable, and fair results. Test-takers must have access to an electronic device (e.g., desktop, laptop, or tablet) that is compatible with the exam delivery software. Likewise, test-takers must have a stable and secure internet connection. The format and content of the test should be amenable to online delivery, which is particularly critical for online screen visualizations of reading passages, images, charts, figures, and geometric shapes. In order to produce valid scores of student performance, the online examination should be designed so that test-takers are tested on the content of what is being assessed, rather than on their familiarity with the testing software or on the use of a specific electronic device.

Remote proctoring is also key to monitor test takers’ behavior to prevent malpractice; for example, some online exam proctoring software can access test-takers’ webcam, microphone, browser activity, and keyboard and mouse to monitor their behavior during the exam, and any suspicious behavior is reported to test administrators for review. 

While all of these factors are necessary to ensure proper administration of online at-home high-stakes exams, many developing countries would require significant targeted investments to meet these conditions. The following five considerations for moving high-stakes exams to an online at-home format, while not providing an exhaustive list, point to the need for significant reflection in making this transition, particularly for developing countries.

First, in many developing (and even developed) countries, not all students have access to an adequate device or internet connection at home to take an online exam. Moreover, students living in overcrowded households may not have the appropriate or sufficient space or conditions to take an exam that will likely be very consequential for their futures. To address this reality, some local governments and organizations in the United States have implemented laptop loaner or donation initiatives and facilitated the availability of public internet hotspots (while maintaining social distancing) to support test-takers who are not able to take online exams at home. 

Second, even with the right infrastructure and space at home, software malfunctions may occur. Adequate time and effort are needed for software testing, providing troubleshooting guidelines, and ensuring the availability of real-time technical assistance prior to and during the online test administration. With regard to remote proctoring in particular, if an online automated proctoring software malfunctions, trained proctors must be on standby and ready to monitor students’ behavior during the online examination remotely. 

Third, remote proctoring has faced legal issues regarding data sharing, confidentiality of personal information, and the use of technologies for surveillance of students’ behavior. In the Netherlands, for example, legal complaints have been made, arguing that online proctoring goes against students’ privacy rights. The Amsterdam District Court has ruled that the use of proctoring software does not constitute a violation of students’ privacy; however, the Court’s decision reaffirmed the importance of proctoring software providers’ compliance with European privacy laws to protect the collection and sharing of students’ personal information. 

Fourth, for examinations previously administered in paper format, it is not enough to simply put the existing examination items in an online format without consideration for other critical aspects of the testing process. With the changes in the exam delivery format, items need to be piloted, evaluated, and adjusted accordingly to meet the content and psychometric standards. For example, the change in the delivery format could turn some items easier or more difficult than in paper format, or simply change the resolution and quality of pictures, plots, or other stimuli; these subtle changes can only be evaluated through pilot studies to ensure the quality of the exam and its content.

For exams that require test takers to perform certain tasks to show their mastery of specific skills (for example, to play a musical instrument or to conduct a science experiment), assessment agencies and testing organizations must develop appropriate tasks to allow for the rigorous measurement of these skills through the online testing format.

Fifth, like other learning assessments, high-stakes examinations should adhere to the principles of universal design to allow all students to have equal opportunity to accurately demonstrate what they know and can do. In fact, testing organizations and government assessment agencies in many countries devote considerable resources to ensure that tests can be adapted for students with disabilities. The at-home online versions of high-stakes exams also need to ensure that all students can be provided with necessary accommodations; therefore, online proctoring needs to consider adaptations to the individual needs of students with disabilities, such as extended time to take the exam, the use of assistive technologies, and the live presence of professionals that can support students to read the questions and record their answers if needed, among others.

Certainly, the challenges of online exam administration are not trivial, and more issues will continue to be identified now that more and more countries and organizations explore online solutions for assessing what students know and can do. As governments and testing organizations work to enhance the assessment solutions they offer for high-stakes decision-making, it will be imperative to ensure that any solution meets the validity, reliability, and fairness properties of a high-quality assessment, especially if it is expected to have a significant impact on students’ lives.

Disclaimer: Resources in this article are provided for informational purposes only. The World Bank does not endorse any of the tools, companies, or applications mentioned in the article.