Two weeks ago, there was a lively discussion on Twitter about this paper by Bursztyn et al., which is about the persistence of political engagement among students in Hong Kong who were given monetary incentives to attend the annual July 1 pro-democracy protests in 2017. David also covered the online discussion here in our Friday links. The paper was a first draft, submitted to the NBER Summer Institute (fooling me into thinking that it was an NBER WP), but quickly found its way into online discussions.
You can read the online criticism and support for the study in the links provided, but the gist of the former was the presence of risks to the students and their families – of imprisonment, of relational repression, of future harassment via the use of facial recognition software, phone tracking, etc. It is certainly not hard to Google these protests and find news of arrests of protesters and protest organizers - in the year in question (2017) and in other years. Using local knowledge on the exact number of arrests of ordinary protesters on the actual day of the protests, rather than of organizers and movement leaders, and on expected attendance, it is possible to put some probabilities on risk of arrest, being held without charges, or emotional or bodily harm. This would allow the researchers to assess whether the risk to study participants is minimal or not. Future harm to students (or their families living in mainland China or colleagues of PIs of this study from other institutions traveling to China) is harder to assess and depends on a bunch of assumptions, including whether the regime is likely to backslide into harsher authoritarianism in the near future, how long a memory it has, etc. I can see reasonable researchers in economics or political science with decent knowledge of the study context disagreeing about the size of these risks.
But, actually, for the first part of this blog, none of this is the actual point I want to make. What I think is regrettable is that the entire brouhaha over this paper could have been avoided if the paper simply had a short section, titled “Ethical considerations, risk assessments, and IRB protocols.” When I wrote to the authors of the paper to suggest that such a section might be useful, they sent me an ethics statement that I am able to share with you here. Furthermore, in further email exchanges, they informed me that they will incorporate the content of this statement into a new ethics section and post an updated working paper. My guess is that had such a section been a part of the draft paper from the beginning, even the harshest critics would have struck a different tone. It is likely that some people would still argue that the risk to the subjects is more than minimal, but the situation is very different if all the cards are transparently on the table and we can argue about our assumptions and expected probabilities of risks, and learn the questions (hopefully serious enough) IRBs asked, the PIs’ answers to those questions, exact study protocols, and perhaps even about our risk aversion levels.
So, lesson #1: If you are conducting an RCT, please include a sub-section on ethical considerations:
- Devote one sentence to routine data collection, informed consent issues;
- Provide IRB approval numbers, but please also provide a link to a document that contains your detailed study protocols (possibly your IRB submission) or include this in an appendix; and
- Discuss in detail more thorny ethical issues that have been covered in the study protocols, provide an assessment of risks, compare them to reasonable benchmarks, give a sense of equipoise and the importance of the study to provide benefits by generating new knowledge…
If you have a really benign intervention (and data collection methods) with negligible risks, this section can be really short. But, it feels better to have a section like this be the default going forward, rather than the current status quo of authors assuming that everything is OK (because they went through an IRB) or that the audience should trust them that they did the right things…
In Part #2, I will take up bigger picture questions that came up during the discussion of this paper, such as:
- Is it OK for researchers to conduct an intervention if it is legal for others to do so?
- Is paying people money to do X coercive? If so, when? How much is too much?
- Researchers “studying” vs. “doing” development…
Stay tuned…
Join the Conversation