A new study presented at Conference on Retroviruses and Opportunistic Infections found that a community intervention (a package that improves take-up, provides community engagement, and post-test support) reduced HIV incidence by 14% at the community level compared to standard-of-care VCT (abstract here). The effect was larger among a sub-group of young females. This is exciting news as the track record of standard VCT approaches in HIV prevention is mixed at best. The researchers, institutions, and funders should be congratulated.
However, the study leaves the reader wanting in two aspects that I keep seeing repeated in biomedical trials.
First, the study design does not allow us to separate the effects of the individual components of this intervention.The intervention includes community mobilization, easy community access to VCT, and post-test support services.Was it the increase in the number of people who got tested that caused the improvement in sexual behavior and onward transmission of HIV in the treatment communities? Was it the community discussing HIV more openly? Or, was it the post-test support, particularly for those who tested positive that caused the improvement? Because the intervention was a package, we cannot say.
The explanation I hear on why people are so enamored by intervention packages is two-fold. First, because it is difficult to demonstrate significant effects on HIV, you want to pack your intervention with all the efficacious treatments known/suspected. This is like combination drug therapies for a disease. But, usually in those cases, such as early days of cancer research, researchers had found out that those drugs individually were not effective, so the combination trialsgave them the interaction effects of individual interventions when combined. The second explanation I get is that Phase III trials are expensive (this one was 34 communities in 4 countries), so you don’t have the luxury to have multiple treatment arms. (Note, I am surprised that the authors are able to detect a 14% decline in incidence with any statistical confidence – abstract states p-value of RR=0.861 to be 0.08 – given that they’re comparing means in 17 treatment communities with 17 control ones. My impression is that such studies are usually powered to find effects of 50% or more – looking forward to the publications…)
These may be valid points, but it cannot be the case that we never are interested in the effectiveness of the components of a program, which is critical for cost-effectiveness. Suppose that the program only increased VCT take-up and had no other components. Would we have seen the same effects? Likely not. Or, alternatively, what if intensive post-test support was provided to people who got tested, but in whose communities there was no difference in the percentage and demographics of who got tested? These seem like important questions to me, the answers to which are not provided in studies like this.
Second one is a bigger pet peeve, as it is a mistake in the interpretation of the data. The link above quotes one of the co-PIs of the intervention stating the following: “In addition… the trial found that men who learned they were infected reduced their overall number of sexual partners by one-fifth and reduced their concurrent partnerships by almost one-third.”
That statement assumes two things not in existence: first, that we can isolate the effect of learning one’s status on their subsequent sexual behavior; and, second, that we observe a comparable group of HIV-positive men at baseline.
Instead, let me suggest two slightly tweaked statements and the accompanying study designs that would have allowed the researchers to make them:
1. You want to make a causal statement about the effect of learning one’s status on sexual behavior: In this case, you could have designed an intervention that only increases take-up or learning one’s status (such as Thornton 2008). Then the intention-to-treat (ITT) effects could be estimated by comparing entire treatment and control communities, and the statement on the effect of learning one’s status on risky behavior could be obtained with a local average treatment effect (LATE) using instrumental variables (IV). In the study design here, the randomized assignment into the combination treatment cannot be used as an instrument for learning one’s status because the intervention can affect sexual behavior through its other components. So, it is OK for the researchers to compare ITT effects on self-reported sexual behaviors (for whatever they are worth) between treatment and control communities, but that is the effect of the entire package – not just that of finding out HIV-positive status.
2. You want to make a causal statement about the effect of this entire treatment on men who were HIV-positive at baseline: In that case, you need to know the HIV status of your entire sample at baseline. Then you can limit your sample to those who were HIV-positive at baseline and rerun your impact analysis. Any effects found are the effects of the entire treatment on people who were HIV-positive at baseline. The difference between the two treatment arms is that a larger group of those in the control group never found out their status PLUS those who found out did not have the support provided by the intervention or experience any changes in community norms or attitudes. In this study, there was only a behavioral assessment at baseline but no baseline HIV testing, so the basis for the above statement is presumably a comparison of people who were tested through VCT in each study arm, which is not a valid comparison because the composition of those groups is determined by treatment status.
(The researchers may have used HIV incidence assays at the 36-month follow-up for everyone in the sample, which is something recent. I am not very knowledgeable about these, but if these assays can tell exactly who was infected with HIV 36 months prior to the study and who was not, then the problem in item #2 would be solved as the researchers would know the baseline status of the entire sample, not just those who got tested through VCT. The forthcoming publications will hopefully shed light on these issues.)
The point here is that you cannot do sub-group analysis on sub-groups, the compositions of which were affected by the treatment. You could do men/women, adolescent/young/adult, etc., but you cannot break the population down by who learned their status and who did not. This type of error, in an otherwise impeccably designed Phase III trial, is not without consequence. It’s easy for the consumers of this research, which includes the likes of me, to read the results and say, “when men find out they’re infected with HIV, they reduce their risky sexual behavior.” That statement may or may not be true, but, as far as I can tell, it is not a finding that this study can claim…
If the authors know the baseline status of their entire follow-up sample of randomly selected individuals from these communities, then they can perhaps state the following: “Among men who were HIV-positive at baseline, our intervention package significantly reduced the number of self-reported sexual partners and concurrent partnerships.” Given that the researchers seem to be arguing a channel of impact due to reduced transmission from infected men that caused reductions in the incidence of HIV among young women, I am curious to find out more about the channels and look forward to reading the forthcoming publications. Interventions for men in this field have long been lagging behind and it would be good to know whether this particular explanation is robust. Regardless, it is good to know that some combination of increased HIV-status awareness, community mobilization, and post-test support can reduce HIV incidence in communities – even if by a modest 14%.
[Caveat: this post is based on the links above, plus a reading of the published study protocols. I have not yet seen articles that describe these findings and very much look forward to reading them.]
Join the Conversation