Examining public examinations around the world

Many of us have memories of studying for a high-stakes examination that would determine whether we graduated from school or got into the university or course of our choice. Depending on how we did, we may have more, or less pleasant associations with the experience.

In October, the World Bank’s Education Global Practice released a new book on the topic of examinations that is sure to be of interest both to those who design examinations and those who take them. We commissioned this book in response to requests for a closer look at the gatekeeper role played by examinations in many education systems, which has huge implications for the degree to which students from different backgrounds have access to scarce educational and economic opportunities. We also wanted to better understand the ways in which examinations are evolving due to technology, more diverse student populations, and changing social and economic conditions. The result is Public Examinations Examined.

Authored by former World Bank staff member, Vincent Greaney, and consultant, Thomas Kellaghan, the book addresses the critical issues facing public examination systems around the world from equity concerns to cheating to the inclusion of students with special needs. I had the chance to talk with Vincent about the book’s key messages, which have a lot of relevance for developed and developing economies.

Marguerite: Why are public examinations so important and what functions do they serve?

Vincent: Public examinations serve many important functions. They certify or assess student competence based on some agreed-upon standards and award successful candidates with diplomas or certificates. They provide a basis for considering candidates for selection to higher levels of education, such as senior levels of secondary school or tertiary education, and for employment. Because of the high level of importance attached to examinations, they help motivate both students and their teachers. In many education systems, they are important sources of income for teachers who provide supplementary tuition or ‘shadow education’. In addition, examinations facilitate administrative control and the promotion of national homogeneity in educational standards and practices through helping to ensure that similar content is taught throughout the education system.

People often confuse public examinations with national or international large-scale assessments, which have quite different objectives. These assessments are primarily concerned with providing policy makers with information on the overall levels of learning in an education system as opposed to the learning levels of individual students. This information is usually derived from a sample of student performance in a limited number of subjects, generally language, mathematics, and science. When well designed and implemented, national or international large-scale assessments can be used to monitor changes in overall achievement levels over time. In some instances, examinations have been used to monitor trends in educational achievement, a dubious practice in most instances given that examination questions change from year to year. Some systems have also used examination results as a mechanism for holding schools accountable for their students’ achievement, without giving due recognition to the many school and non-school factors that contribute to student achievement. 

MC: It’s evident that public examinations are critically important aspects of many education systems. Can you talk about some of the challenges confronting examination systems?

VG: One of the biggest challenges facing many examination systems is the difficulty of reforming them. Many countries must confront the reality that the chances of success on their public examinations are often loaded in favor of, or against, students of a particular gender, place of residence, socio economic background, type of school attended, and ethnic or language group membership. While these inequities can create the impetus for reform, examination systems first need to confront well-established, embedded traditions that have evolved over the decades, and will need time to move away from a “We’ve always done it this way” mindset. Possible opposition to reforms may come from students and parents who prefer the status quo as well from those who are likely to suffer financial loss, such as examination staff or private tutors. Examination agencies need time to become familiar with new technologies and administrative processes; “quick fixes” are unlikely to bring about effective improvements to existing systems. Additional finance will be required to support reforms that develop technical capacity, and especially for initiatives that introduce computer adaptive testing (as in Georgia), school-based assessment (as in Queensland, Australia) , oral and practical assessment (as  in Ireland), and examinations administered to candidates with disabilities and diverse educational needs (as in the United Kingdom). Increasingly, examination agencies are beginning to appreciate how ill-equipped they are to address examination malpractice reflected in inappropriate behavior such as cheating, bribery, misuse of results, collusion with other candidates and supervisors as well as obtaining information from external sources by various means, including by modern technology. Chinese authorities, for instance, revealed that some examination candidates used highly-sophisticated scanners and hearing devices concealed in reading glasses to obtain external assistance during a key public examination.

MC: While examination reform may be a challenge for countries, it’s also a necessity. What factors do you think should be considered by countries when planning a program of examination reform?

VG: The success of an examination reform will depend to a considerable extent on the willingness of the political and educational authorities to embrace change, and on the likely impact of the proposed reform on public confidence. Reform programs should be sensitive to existing strengths and weaknesses and should consider a long-term strategy for improving assessment procedures. Students and teachers should be given enough time to become accustomed to new examination processes, including pilot testing of unfamiliar processes and formats. Education systems may also need time to modify curricula and textbooks and discuss the likely impact of new forms of examination results with users such as tertiary level institutions. Staff may require training in modern assessment approaches, including methods of ensuring that examination papers adequately reflect the official curriculum, that scoring systems are reliable, and that results are produced in a timely manner. Governments should consider programs to cover the examination costs of needy students, and other measures to remove existing inequities. Policy makers also can use a reform agenda as an opportunity to discontinue costly public examinations (such as primary school leaving exams) in cases where most students advance to the next highest level of the educational system. Finally, teachers, providers of pre and in-service teacher education programs, curriculum specialists, and others should capitalize on the detailed analyses of examination results, which give them valuable insights into common student misunderstanding and errors.   

MC: Your book contains a lot of very useful information about public examinations and I would encourage World Bank staff, country policymakers, education practitioners, and others to check it out. Before we finish, do you have any final advice to share?

VG: Education systems should have realistic reform goals, which consider existing strengths and weaknesses. Pilot reform initiatives can be particularly useful; Russia carried out many pilot programs before it introduced its Unified State Examination. Over-ambitious programs designed to meet the time demands of a project cycle or of an unrealistic government commitment can be counterproductive. Shakespeare’s observation is apt: “Wisely and slow; they stumble that run fast.” In addition, we must be careful that examination scores do not become the goal of teaching as this can have a negative effect on teaching and on the quality and nature of student learning. Carefully thought-out examination reform initiatives, which have broad stakeholder support, can be very beneficial especially if they help guide teaching, direct attention to important topics and skills specified in the curriculum, and help improve student certification and selection processes.