Public randomization ceremonies

With Jake Robyn* and Gaston Sorgho**

Randomization might- at first – sound like a scary word for health policy makers and professionals. They read medical journals and know from their training that randomized trials are scientifically rigorous designs to evaluate the impact of a program. But their first inclination might be to prefer to have the randomized trial in somebody else’s backyard. Randomization seems politically difficult. How to explain it to the people who will have to wait for the new intervention? Will it not create a backlash with the people who are randomly assigned to the control group? How will the population be convinced that the random allocation was fair and that there were no back room deals?

Our experience in many countries is that public randomization ceremonies are an excellent platform to build support for randomization and for the entire impact evaluation process. In Cameroon, we organized public randomization ceremonies in three Regions to assign health facilities to four study groups in an impact evaluation of performance-based financing (PBF) in the health sector. Held in the regional capitals and combined with the official launching of the project in each Region, we invited representatives of each facility, district health management teams, and local government, who all took part themselves in the randomization. Each of the randomization ceremonies received close oversight from the central and regional levels of the Ministry of Public Health. This made the randomization process completely fair and transparent to all health facilities participating in the study.

We made a video of those ceremonies in Cameroon illustrating the process and the reception by health facility representatives. The film includes images from all three regions but primarily focuses on the ceremony held in the Eastern Region of Cameroon in July 2012. The video provides context for the evaluation – for those readers interested in just seeing the randomization in action, skip ahead and watch from minute 6:32 to 13:15, and from minute 2:34 to 3:36 for the preparation of the randomization blocks.

The video is pretty much self-explanatory, but here is some additional background on the PBF project in Cameroon and its impact evaluation which will provide some context. With the objective of improving maternal and child health outcomes, Cameroon has recently started to implement performance-based financing (PBF) in the health sector. 

Performance-based financing is as a mechanism by which health providers are, at least partially, funded on the basis on their performance, usually measured both in terms of quantity and quality of services delivered. Performance-based financing can be contrasted with the line-item approach, which finances a health facility through the provision of inputs (e.g. drugs, personnel). The Ministry of Public Health, in collaboration with the World Bank, has launched a pilot implementing PBF in public and private facilities across 26 districts in the Littoral, North-West, South-West and East regions of Cameroon covering a total population of approximately 2.8 million. The impact evaluation is conducted in 14 of these districts.

PBF does not only introduce a new incentive mechanism for health facilities and health care providers. It also comes with reinforced supervision and increased managerial autonomy. In addition, PBF will also increase the amount of resources available for the health facility. So, the full PBF package implies performance incentives, an increased budget and stronger supervision and larger financial autonomy. The impact evaluation is designed to isolate the effect of these different components, and has the following four groups:

1.  T1: the treatment group that gets the full PBF program
2. C1: a control group which experiences the same level of supervision and managerial autonomy and an equivalent amount of per capita financial resources as in T1, but those resources are not linked to performance.
3. C2: no additional resources compared to the status quo, but enhanced supervision and monitoring will be performed
4.  C3: Status quo, with no interventions.

The comparison between T1 and C1 can isolate the effect of performance-based incentives. The comparison between C1 and C2 allows isolating the effect of increased financial resources and managerial autonomy. The comparison between C2 and C3 allows focusing on the effect of enhanced supervision and monitoring. Comparing the results in T1 with those in C3 will show the impact of the full PBF package.

Because of sample size requirements (we needed about 50 units in each of the four groups), the random allocation between the 4 groups had to be done at the level of the health facilities.  To make sure that all three regions and all type of facilities (public vs. private) were equally represented in each four groups in each region, we blocked the randomization at the regional level and within regions by health facility characteristics. In one region, not shown in the video, we further blocked by rural vs. urban. We took care of the “blocking” by regions by organizing three different public randomization ceremonies, as described above. The blocking by type of facility was done by conducting the randomization process within each health facility block in turn. As shown in the video (minute 2:34 to 3:36), we first put the names of the private and the public health facilities in two different bowls. Next, we first proceeded with the randomization for the private health facilities (starting at minute 8:27), and finally randomized the public facilities into the four groups (starting at 9:40).

*Jake Robyn is a Health Specialist in the Africa Region of the World Bank.

**Gaston Sorgho is a Lead Public Health Specialist in the Africa Region of the World Bank.