Berk, thanks for kicking off this important discussion. I’d like to focus on your 4th point, about phased-in designs. Two things.
First, I feel I am missing something in the argument about whether delayed treatments with full information can still be unethical. Usually, the appealing premise of a phased-in design is that there is some resource constraint that would prevent simultaneous scale-up in any case. In this scenario, no matter how heavy the burden of waiting, there will be to be some rationing. In which case, why not randomization rather than something else, like patronage? Then things get odd. The suggestion seems to be that we may know, ex ante, that at least some types of people (elderly, immune-compromised) will benefit greatly from immediate receipt of the treatment. In which case, we are not in equipoise and whether an RCT (or at least unconditional randomization) is appropriate in any case. Things, of course, get trickier when a resource constraint is not binding simultaneous scale-up.
Second, I feel we should reflect on the purpose and ethics of a phased-in design, especially one with full information. I blabber on about it here as well: http://hlanthorn.com/2014/01/24/something-to-put-in-your-friday-pipeline-and-i-am-not-so-sure-about-phase-inpipeline-designs/
Again, a resource constraint may make it politically acceptable for a governor to say that she will roll-in health insurance randomly across the state, which can allow an opportunity to learn something about the impact of health insurance. So, she stands up and says everyone will get (this) health insurance at some point and here’s the roll-out schedule.
But the reason for making use of this randomization is to learn if something works (because we genuinely aren’t sure if it will, hence needing the experiment) and maybe to have ‘policy impact’. So what if what is learnt from comparing the Phase I and Phase II groups is that there is no impact, the program is rubbish or even harmful? Or, at a minimum, it doesn’t meet some pre-defined criterion of success. Is the governor in a position to renege on rolling out the treatment/policy because of these findings? Does the fine print for everyone other than those in Phase I say “you’ll either get health insurance, or, if the findings are null, a subscription to a jelly-of-the-month club”? In some ways, a full-disclosure phased roll-in seems to pre-empt and prevent policy learning and impact *in the case under study* because of the pre-commitment of the governor. I find that phased roll-in designs without a plan to pause, analyse, reassess and at least tweak the design between Phases I and II to be ethically troubling. I’d be interested in your thoughts.
Also, I would like to echo the point that the farther from a laboratory you get, the less helpful is the present IRB system.
Thanks again for engaging!