Making policy evidence-based: It’s also hard in developed countries…

|

This page in:

Update: As if on cue, the Washington Post published an article (on January 19, 2012, 3:42 PM EST) that says:

A new government study suggests a lot of teenage girls are clueless about their chances of getting pregnant. In a survey of thousands of teenage mothers who had unintended pregnancies, about a third who didn’t use birth control said the reason was they didn’t believe they could get pregnant. Why they thought that isn’t clear.

Another finding from the same study:

Nearly a quarter of the teen moms who did not use contraception said they didn’t because their partner did not want them to. That suggests that sex education must include not only information about anatomy and birth control, but also about how to deal with situations in which a girl feels pressured to do something she doesn’t want to.

Recently, the Obama administration went against its Food and Drug Administration’s (FDA) recommendation and stopped the Plan B morning-after pill from moving onto drugstore shelves next to the condoms. The administration, including the Health and Human Services (HHS) secretary Sebelius and the president himself, who backed the decision to overturn the FDA’s recommendation, took enormous heat (see here and here). The move has been depicted as politics overruling science. However, there was also a lack of scientific evidence – just not where all the critics of the administration were pointing: it had nothing to do with either the safety of Plan B or the issue of whether teenagers could use it properly.

Let’s start with the argument that the science is all there. I agree. But, which science are we talking about here? The case is a strict drug approval case and under the rules, FDA has to establish (a) the drug’s safety, and (b) whether children 11 or older can use it safely. And, as far as we can tell, they have done that. I could not find the Center for Drug Evaluation and Research’s (CDER) report for this round of the approval process, but the one from 2005 by the American Academy of Pediatrics (AAP) is informative. It clearly establishes safety and the knowledge of use. However, there is only one paragraph on potential disinhibition effects, with 3 citations out of more than 100 studies cited in the six-page report, all of which are based on self-reported data on frequency of unprotected coitus and condom use. That is it.

Perhaps the AAP and the CDER can be excused for not dwelling too much on this. Disinhibition is not a drug approval issue. But, it is the issue some of the opponents of Plan B are worried about. Other than drug safety, the relevant questions are: (a) does Plan B actually decrease unwanted pregnancies among teen-age women; and (b) might it cause a rise in sexually transmitted infections among the same group? The use of this emergency contraceptive by young girls is a legitimate public health question that deserves to be examined. Had the administration addressed these questions, it would both have engaged the opponents in the correct manner and avoided infuriating the people who want this drug in drugstore aisles today. The policy choice of drug approval or not is a narrow binary one. A richer policy discussion could have jointly considered approval of the drug along with parallel efforts aimed at disinhibition, but that discussion seems to have been precluded by the sequence of decisions necessitated by the institutional structure as well as fractured institutional responsibilities (I thank my colleague and fellow blogger Jed Friedman for pointing out this last point).

So, what is the science on the introduction of Plan B and STIs? While the 2005 AAP report has no citations of work from other disciplines such as demography or economics, there is actually some work on the effect of emergency birth control (EBC) on pregnancy and STIs. This paper in the Journal of Health Economics, published last year, suggests that the introduction of EBC over the counter and free of charge in England to adolescents younger than 16 may have had no effect on the conception rates of teenagers while leading to an increase in the prevalence of STIs. The authors also cite the works of economists, conceptual and empirical, going back a while. The summary of the evidence from these studies, the number of which is definitely less than ideal, is that there is no convincing evidence that EBC reduces pregnancy rates and, if anything, they might increase STI rates among the population of teenage girls.

The idea that adolescents may rationally respond to the existence of insurance against pregnancy in the form of easily accessible morning-after pill by being less careful about unprotected sex, seems to be lost on the crowd “mad as hell” at Obama (see this Slate.com article for an example). People eat worse and exercise less when they’re on statins, so why do we think teenagers may not respond to the availability of something that takes pregnancy out of the equation?

(As a side note, it is actually hard for me to imagine that Plan B would not decrease unwanted pregnancies. For this to happen, the disinhibition effect would have to be quite large, along with a non-negligible failure rate among these additional unprotected sex acts – either due to failing to use Plan B afterwards or due to Plan B not being 100% effective in preventing pregnancies. If girls have more unprotected sex because Plan B is readily available, then you’d think that they’d then use it as prescribed, leading to no adverse effects on pregnancies. The lack of evidence on reduced unwanted pregnancies is likely a study design issue having to do with power, etc., rather than the lack of a real effect. However, obviously, we cannot make the same argument about STIs, the prevalence of which can reasonably increase.)

The problem is this. These days, unfortunately, unprotected sexual activity has two potential consequences: pregnancy and sexually transmitted infections. Despite this being the 21st century, we still do not have something other than condoms that prevents both (when someone finally invents an easy to use drug that stops sperm as well as bacteria and viruses dead on their tracks, they will make a fortune). So, if a teenage girl (or an adult woman) wants to protect against both, she has to abstain or use condoms. In cases where a condom is not immediately available or not desired by either party, Plan B might make it easier to forego protection - it is an empirical question. If we had the answer, in the form of reliable statistics on STIs, then the administration could engage the opponents of Plan B better – with the science that is pertinent here.

I don’t intend to downplay the costs of Plan B here, which come in the form of both side effects and monetary costs. Unlike in the England study, Plan B is not cheap in the US, let alone free. These do weigh against the idea that teenagers would use it as non-emergency birth control. But, again, it is an empirical question – one that can be studied. And, the critics of the administration are right: the stress of having to run around for a prescription after a broken condom the morning after (or worse, rape) or to deal with pharmacists who won’t sell you Plan B are unpleasant to even contemplate.

The administration should have allowed the FDA to approve Plan B. But, the concern that STI incidence might increase among teenagers with the availability of this drug may be legitimate. Instead of implicitly treating the opponents of Plan B as ignorant conservatives who don’t care about science and infuriating the supporters by interfering in a drug approval case, the HHS should have stayed out of the FDA’s business and then proceeded with what it is charged to do: protect public health. In this case, it would have meant addressing the issue of whether the increased availability of Plan B for teenagers might increase the prevalence of STIs among them by increasing unprotected sexual activity. It would have meant collecting evidence to have a more well-rounded policy on teen pregnancy and sexually transmitted diseases: evidence based policymaking, not the unnecessary brouhaha that we witnessed.

P.S. I came across this piece from New Humanist on evidence-based policy making this morning: http://newhumanist.org.uk/2721/burden-of-proof-should-evidence-determine-policy?

 

Authors

Berk Özler

Lead Economist, Development Research Group, World Bank

Tim Ogden
January 19, 2012

Berk,

good on you for picking an example with nuance and strong arguments on both sides.

But there is an immediate example with far less nuance where evidence-based policy is apparently also impossible.

Yesterday came news that Gov. Christie of NJ signed a unanimous resolution of both houses of the NJ legislature that the state would reject the recommendations from the medical review board that PSA testing for prostate cancer be dropped. Exhaustive evidence has shown that PSA is only useful in catching prostate cancer that progresses so slowly it is not worth treating. The dangerous prostate cancer, which is highly aggressive and invasive progresses so quickly that PSA doesn't improve survivability.

Bottom Line: huge waste of money that does no one any good.

Policy: We don't care.

A powerful argument for Pritchett's "very good answers to questions that don't matter" position.

(Though thanks for responding to his recent CCT posts!)

Berk Özler
January 19, 2012

Thanks Tim, I had not heard of the PSA case -- it is a good example.