Syndicate content

Public randomization ceremonies

Damien de Walque's picture

With Jake Robyn* and Gaston Sorgho**

Randomization might- at first – sound like a scary word for health policy makers and professionals. They read medical journals and know from their training that randomized trials are scientifically rigorous designs to evaluate the impact of a program. But their first inclination might be to prefer to have the randomized trial in somebody else’s backyard. Randomization seems politically difficult. How to explain it to the people who will have to wait for the new intervention? Will it not create a backlash with the people who are randomly assigned to the control group? How will the population be convinced that the random allocation was fair and that there were no back room deals?

Our experience in many countries is that public randomization ceremonies are an excellent platform to build support for randomization and for the entire impact evaluation process. In Cameroon, we organized public randomization ceremonies in three Regions to assign health facilities to four study groups in an impact evaluation of performance-based financing (PBF) in the health sector. Held in the regional capitals and combined with the official launching of the project in each Region, we invited representatives of each facility, district health management teams, and local government, who all took part themselves in the randomization. Each of the randomization ceremonies received close oversight from the central and regional levels of the Ministry of Public Health. This made the randomization process completely fair and transparent to all health facilities participating in the study.

We made a video of those ceremonies in Cameroon illustrating the process and the reception by health facility representatives. The film includes images from all three regions but primarily focuses on the ceremony held in the Eastern Region of Cameroon in July 2012. The video provides context for the evaluation – for those readers interested in just seeing the randomization in action, skip ahead and watch from minute 6:32 to 13:15, and from minute 2:34 to 3:36 for the preparation of the randomization blocks.

Impact Evaluation of Results Based Financing

The video is pretty much self-explanatory, but here is some additional background on the PBF project in Cameroon and its impact evaluation which will provide some context. With the objective of improving maternal and child health outcomes, Cameroon has recently started to implement performance-based financing (PBF) in the health sector. 

Performance-based financing is as a mechanism by which health providers are, at least partially, funded on the basis on their performance, usually measured both in terms of quantity and quality of services delivered. Performance-based financing can be contrasted with the line-item approach, which finances a health facility through the provision of inputs (e.g. drugs, personnel). The Ministry of Public Health, in collaboration with the World Bank, has launched a pilot implementing PBF in public and private facilities across 26 districts in the Littoral, North-West, South-West and East regions of Cameroon covering a total population of approximately 2.8 million. The impact evaluation is conducted in 14 of these districts.

PBF does not only introduce a new incentive mechanism for health facilities and health care providers. It also comes with reinforced supervision and increased managerial autonomy. In addition, PBF will also increase the amount of resources available for the health facility. So, the full PBF package implies performance incentives, an increased budget and stronger supervision and larger financial autonomy. The impact evaluation is designed to isolate the effect of these different components, and has the following four groups:

  1.  T1: the treatment group that gets the full PBF program
  2. C1: a control group which experiences the same level of supervision and managerial autonomy and an equivalent amount of per capita financial resources as in T1, but those resources are not linked to performance.
  3. C2: no additional resources compared to the status quo, but enhanced supervision and monitoring will be performed
  4.  C3: Status quo, with no interventions.

The comparison between T1 and C1 can isolate the effect of performance-based incentives. The comparison between C1 and C2 allows isolating the effect of increased financial resources and managerial autonomy. The comparison between C2 and C3 allows focusing on the effect of enhanced supervision and monitoring. Comparing the results in T1 with those in C3 will show the impact of the full PBF package.

Because of sample size requirements (we needed about 50 units in each of the four groups), the random allocation between the 4 groups had to be done at the level of the health facilities.  To make sure that all three regions and all type of facilities (public vs. private) were equally represented in each four groups in each region, we blocked the randomization at the regional level and within regions by health facility characteristics. In one region, not shown in the video, we further blocked by rural vs. urban. We took care of the “blocking” by regions by organizing three different public randomization ceremonies, as described above. The blocking by type of facility was done by conducting the randomization process within each health facility block in turn. As shown in the video (minute 2:34 to 3:36), we first put the names of the private and the public health facilities in two different bowls. Next, we first proceeded with the randomization for the private health facilities (starting at minute 8:27), and finally randomized the public facilities into the four groups (starting at 9:40).

*Jake Robyn is a Health Specialist in the Africa Region of the World Bank.

**Gaston Sorgho is a Lead Public Health Specialist in the Africa Region of the World Bank.

Comments

Submitted by Conner on

Interesting stuff. Any thoughts on how to incorporate public randomization events into a randomized phase-in without inducing anticipation bias? Hopefully the answer isn't just "lie about the fact that the controls will later receive the program." In my experience, government ministries and NGOs alike are much more open to randomized phase-ins than completely excluding certain communities from programs.

Submitted by Damien on

Thank you very much for your comment and question. In the case of Cameroon, the health facilities that were randomized in one of the control groups will receive the full PBF package (potentially adapted and fine-tuned based on the impact evaluation results) after 2 years. This is planned and budgeted. I agree with your point that this might create some “anticipation bias”, but in such randomized phase-ins, it is unavoidable that the control facilities will be aware of the program and of the fact that they are scheduled for the next phase. Given that, we think that having the entire process, including the randomization, being explained and performed in public builds support for the impact evaluation by showing it is fair and transparent.

Submitted by Adam on

Couldn't find this study in any of the trial registries including the new AEA/ JPAL trials registry. Any thoughts on the pro and cons of design registration at this stage?

Add new comment