Last year the British Medical Journal published the results of an impact evaluation of local immunization campaigns with and without incentives in rural India. Full immunization rates were very low in the study area (2%) and the researchers wanted to test two nested approaches to improving participation in immunization campaigns. The first approach was to introduce periodic immunization camps that would guarantee service on select days. The second was to introduce small incentives at the camps in the form of food and household goods that roughly approximate the opportunity cost of camp participation. Later that same year the BMJ then published a letter accusing the study of violating ethical guidelines for research.
First a summary of the study findings. The authors Abhijit Banerjee, Esther Duflo, Rachel Glennerster, and Dhruva Kothari determined that regular immunization camps raise the full immunization rate from 6% (as found in control communities) to 18%. Adding low-powered incentives to the camps further raises the immunization rate to 39%. These gains, while still far from achieving universal coverage, are substantially higher than the rates in the “health-care-as-usual” situation in control areas. Because the immunization rates were higher in the camps with incentives, the cost-effectiveness of such an approach is actually much higher than the camps alone thus suggesting the provision of a small incentive is well worth the additional cost of doing so.
A subsequent letter published in the BMJ by Deepak MG and Anant Bhan calls the study to task on ethical grounds. The letter writers claim the study does not establish new knowledge (i.e. the world already knows that health behavior responds to incentives) and thus is not justified. With this claim the letter writers invoke the principle of Essentiality. This is the first principle listed in the Ethical Guidelines for Medical Research as published by the Indian Council of Medical Research. The principle stipulates that approved research should be deemed absolutely essential after a due consideration of all alternatives. To defend their claim of non-essentiality, the letter writers cite examples of previous knowledge in the form of evaluations of recurring CCT programs in Latin America. While I agree that essentiality is a key principle of ethical research, I don’t see how the study violates this principle. Specifically:
1. The applicability of previous knowledge is far from obvious to the more episodic immunization camps and lower powered incentives adopted by the study in rural India.
2. The study was approved by three independent review boards and was funded by several donor agencies. Presumably all review boards and donors considered the “essentiality” of the study and all agreed that the study should go forward. People may hold opposing views on the “essentiality” of a proposed study, but there is no central board that makes such a determination and that’s probably a good thing. In the decentralized review process the fact that at least one accredited body deems a study to be essential should be sufficient. Indeed the research in question did provide evidence that was used in subsequent policy discussions, as can be seen in this note by Jishnu Das and this longer article by Ramanan Laxminarayan and Nirmal Ganguly.
3. As made clear in the first paragraphs of the published paper, the research question is not just whether immunization camps have an observable effect at all but rather the magnitude of any effect and whether an effect of that magnitude justifies the spending of public resources at the expense of alternative uses. I do not study immunization policy, but it certainly seems legitimate to ask just how effective immunization campaigns are and whether low-powered incentives such as those included in the study are a cost-effective way of improving immunization rates. This is not the first time the validity of policy-oriented research questions has been challenged by medical professionals but this does not make these questions any less essential.
The letter writers do not stop with accusations of inessentiality but also claim that, by virtue of having a control group, this study violates the principle of the Non-Exploitation of Study Subjects as well as violates the norm that all participants should be beneficiaries of the research. What is meant by their claims here?
Again taking the Ethical Guidelines for Medical Research linked above, the Non-Exploitation Principle states that research participants should be remunerated for their involvement in the research and kept fully appraised of all potential dangers from the study. There were no dangers to the control group associated with the study, so the assessment hinges on what is meant by “remunerated”.
The ethical guidelines of the Indian Council of Medical Research are somewhat vague on the meaning of this term, however it does appear at first glance to be a stronger principle than the “Do No Harm” principle stipulated in the seminal 1979 Belmont report. The guidelines state “Participants may be paid for the inconvenience and time spent, and should be reimbursed for expenses incurred, in connection with their participation in research”. So perhaps the remuneration stipulation in the non-exploitation principle should take the form of direct payments? However, at a later point, the same ethics document discusses the conduct of epidemiologic research and concludes:
In a country like India, with the level of poverty that is prevalent it is easy to use inducements, especially financial inducements, to get individuals and communities to consent. Such inducements are not permissible. However it is necessary to provide for adequate compensation for loss of wages and travel / other expenses incurred for participating in the study.
So in one sense, the remuneration claim relates to compensation for time lost as a result of study participation. I don’t know if interview subjects were nominally remunerated for their time spent in interview in this case, but this is a fairly common practice and I would not be surprised if the investigators did indeed do so.
More generally remuneration surely takes the form of knowledge advancement and the possibility that participating control communities will derive future benefits from the research in the form of improved policy. (And these benefits would not be possible without the results of the research in question.) Indeed this relates to yet another principle, the Maximization Principle, as set forth in the same Ethical Guidelines:
Maximization of the public interest where the research is conducted to benefit all human kind and in particular the participants themselves and the community from which they are drawn.
The authors clearly state that assuring immunization is a great challenge in the study communities and surrounding areas – any policy learning from the study is meant foremost to benefit these underserved communities. In sum, as with the first charge of non-essentiality, I don’t see how the second charge of exploitation is valid.
Hopefully this disagreement has illuminated select principles of ethical research as well as their interpretation. The title of the post concerns the ethics of control groups so let’s now ask what should be the ethical standard for the use of control groups. Should the actual control groups benefit in some direct way from the study or is the assurance that any policy gain from knowledge generated by the study be widely shared sufficient? In part this determination relates to the essentiality claim. If a randomized study is already deemed essential to the extent that the internal validity requirement necessitates a control group, then the benefits of increased knowledge and policy learning may satisfy the remuneration claim in the non-exploitation principle. From this view, it comes down to whether a research question is deemed essential by at least one peer review group.
But perhaps researchers can do more. In contrast to no direct provision of benefits for the control group, Osrin and colleagues put forward the principle of “no survey without service” for the use of control groups. Under this principle, control groups should receive more tangible benefits than policy learning, but these benefits need be unrelated to the study question in order to not confound any inference of causal impact. Examples given by Osrin and co-authors of such benefits include supplementary training for health workers on issues not addressed by a particular study, or health information campaigns to the populace again on unrelated issues.
Right now “no survey without service” is not standard practice in socio-economic oriented impact evaluations yet this is an interesting principle that should be more widely considered and instituted where feasible. However I can imagine evaluations where the application of any service or benefit to a control group confounds the inference of program impacts. In these situations, perhaps we again need to return to the essentiality principle – a study deemed sufficiently essential may be justified even if services can’t be provided to the control group (at least during the duration of the study).
Impact evaluations in the social sciences may soon need their own specific ethical guidelines and I wonder if this principle of “no survey without service” is one that should be considered. If so, then researchers will also need guidance as to when this principle can justifiably be suspended.
Join the Conversation