Condition: a situation that must exist before something else is possible or permitted.
Condition: the circumstances or factors affecting the way in which people live or work, especially with regard to their well-being.
When we talk about conditional cash transfers, or CCTs, we’re referring to the first definition: eligible individuals must abide by some prespecified conditions before they can receive the cash transfers. But we can also talk about conditions without which an intervention (say, unconditional cash transfers, or UCTs) won’t affect an outcome (e.g., infant mortality). That refers to the second meaning… People also refer to the latter as mediators and, of course, you can condition interventions on mediators if you think that it will make your policy more cost-effective, more palatable to the public, etc.
A new paper finds that large cash transfers given (in a sort of lump-sum manner) to poor people reduced infant mortality by close to 50% in Kenya. The cash transfers were unconditional – eligible people did not have to do anything to receive them. However, the reported effects seem to have been very much conditional on the following:
· That the transfers overlapped with the birth month (neonatal period);
· That the transfers were given to people who lived close to a physician-staffed facility; and
· That the transfers were given to people who lived in high-saturation cash transfer areas (i.e., in villages surrounded by many other villages who also received cash transfers).
Add to these the fact that Kenya experienced a severe drought during the cash transfer period that substantially elevated infant mortality, you arrive at a result that depended on a lot of “conditions.”
[Quick background context: This paper builds on the experiment reported in Egger et al. (2022). The advantage of that experiment was that, in addition to its large scale (both in coverage and transfer size), it had a two-stage randomization of sublocations into low- and high-saturation of treatment (LS and HS, respectively from hereon), and then villages into treatment or control. That paper found large effects on economic outcomes, which would have been underestimated had it not been for spillover effects detected across villages close to each other. It should be noted that (as it will be important below), in that paper, the spillovers largely accrued to ineligible households in treatment villages, as well as control villages in HS. The spillovers on eligible individuals in HS control villages were small and statistically non-significant.]
That cash transfers can improve child health is well established. We can believe that the effect on infant mortality is large in this study, and yet also believe that we are unlikely to replicate it elsewhere. Here are some circumstances that, I think, might have contributed to the finding of a very large effect:
1. Selection bias: Table 3 shows that the probability of giving birth increased by about 11% during the UCT period. This kind of selection on the extensive margin (fertility) is bad whenever you’re reporting intensive margin effects (mortality, which requires the existence of a person) in an RCT. The authors do what they can to look at selection on observables, but this does not wipe out the concern about biased estimates. This is especially because, one would expect heterogenous treatment effects of UCTs on fertility: all around the world, positive income shocks cause young women to delay pregnancies and marriages, especially in areas with transactional sex, like Western Kenya. So, when I see the 11% increase in fertility during the UCT period, it makes me think that there are hugely heterogenous treatment effects going on with respect to fertility: young women are having less of it – with some of them delaying the onset of childbearing, while others already in relationships are increasing their fertility (or changing its timing) – by a lot more than 11%. Both would work in the direction of pushing infant mortality rates (IMR) down…
2. The drought is self-explanatory: the IMR soars from approximately 30 to 60 in 2017 in LS treatment areas (Figure 1), while cash transfers likely provided a protective effect during this period. This paper by Baird, Friedman, and Schady (2011) shows the effects of negative shocks on IMR using DHS data (remembering fondly the days you could download DHS data). Without this type of massive shock, the effect size would have likely been approximately 5 percentage points (pp), or 12-13 percent, lower (my informed speculation – based on the evidence in the paper).
3. Baseline balance and trends in IMR: Figure 2 shows that HS started from an IMR of around 40, while LS started at 34. LS increased to 40 in the next period - partly because of the drought in 2017. In another study, these figures could have been reversed, resulting in smaller effect sizes.
You put these three circumstances together with the very specific finding (that cash has to be given at exactly the right time to people living near a facility with a doctor) and with what is likely the largest cash transfer experiment we have ever seen (and one that is unlikely to be replicated by governments), and you have yourself this large effect. I am not saying that (the absence of) these conditions would eliminate the beneficial effect of cash transfers on IMR altogether, but they might explain why it is so big…
Puzzling over spillover effects…
Table 1 shows that spillover effects are a big part of the story here (just as proximity to physician-staffed facilities is, as shown in Table 2). In the authors’ own words, they “… find that a meaningful portion of the reported mortality reductions are driven by local cash transfer spillovers.” However, this statement clashes with other evidence they present and puzzles me for three reasons:
1. First, let’s look at Table 1, columns 1-2 (the reduced form estimates, which I prefer because they are simpler without the need for IV): if you are careful, you will notice that the total reduction on infant and child mortality is the same (17-18 children out of 1,000 live births). This implies that conditional on surviving to age 1, the program had no additional effect on child mortality rates (CMR): the control means for IMR and CMR in LS imply that an additional 17 deaths take place between ages 1-5, identical in both control and treatment groups – all the effect comes from the first year, most of which, in turn, comes from the neonatal period (as reported in Figure 3).
a. However, the reporting of direct (own village) vs. indirect (HS spillovers) effects for these two outcomes reveals a puzzle: column 1 shows that the effect on IMR is largely due to spillovers (accounting for about two thirds of the total effect). This is puzzling enough as discussed for other reasons below, but, for now, it is discordant with the evidence presented in column 2, where this picture is reversed: the effect of the program on CMR is primarily due to own transfers (again, accounting for about two thirds of the total effect). I can’t think of a reason why the channel of impact would flip when there is no program effect on CMR conditional on IMR?
2. Second, the likely spillover story is that I am not only richer from my own transfer but also have a much better economic outlook because people around me are also richer, economic activity has increased, and I am benefitting from the multiplier effects. However, the study claims that cash matters only when provided during a very specific (and short) window around birth. If that is true, then why are spillovers so important in reducing IMR when spillovers from other villages would take months, if not longer, to materialize as economic gains (multipliers) for recipient households? This could only realistically happen if transfers were anticipated and recipients could plan (an ambiguity in the study that I discuss below).
a. Figure 6 shows that the program increased the likelihood of delivering at a hospital (not clear whether physician-staffed or not) by 20 pp! Why were UCT recipients in HS areas more likely to deliver in high-quality hospitals than those in LS areas? Was it the fact that they decided ahead of time and planned facility deliveries? Or were they equally likely to plan home births but could get to a hospital in case of an emergency and afford it? Either way, UCT recipients with equally easy access to a facility with a physician could do the same regardless of whether they were in an HS or LS area: what was it about HS that changed this propensity? It is hard to imagine it was economic spillovers within that short a window: how would other people's transfers affect my delivery plans within a month of transfers (or, worse, when we encounter complications at home)?
3. If spillover effects are an important part of the story for IMR and that Egger et al. (2022) showed large spillover effects on economic outcomes on ineligible households, why don’t we see any effects on them on IMR? Table A.5 shows no sizeable spillovers on a combination of eligible HHs in control villages and ineligible HHs. The authors argue that IMR is much lower among ineligible HHs and is, therefore, harder to move. Fair enough… But, on the flip side of this argument, economic spillovers on eligible households in LS areas were negligible in Egger et al. (2022): how are they now affecting IMR (and affecting it so fast)? It’s a puzzle…
A few study weaknesses:
1. One of the things that the authors do is to condition the analysis on a post-treatment variable: namely physician-staffed hospitals. Hospitals are counted by recall (to exist in 2014, i.e., pre-intervention) but counted as staffed (or not) with a physician now (during the survey). This is a no-no in RCT analysis, because the conditioning variable (physician-staffed facility) can be affected by the intervention: if deliveries are costly for the HH, payments are made to the hospital, and doctors benefit from those payments, it would not be unimaginable for doctors to be more present at hospitals in HS areas in response to increased demand for facility-based deliveries: this would not only bias the impact estimates here, but also might come at the expense of other hospitals in LS areas (another type of spillover that seems very likely in an experiment like this). Relatedly, there is evidence (from earlier CCT experiments) that CTs can improve the supply side…
a. Comparing the control and treatment means at the bottom of Table 2 for below median facilities; I am indeed worried about treatment effects on the proximity to a facility.
2. Related to the above, there are several forking paths in the study, which seem to affect the story the authors tell. These include whether to analyze distance to hospitals vs. hospitals with doctors (the former shows no effect); reduced form vs. IV effects as the main analysis (RF often differs than the IV effects); periods during which effects are found (only during the UCT period and only within a very small window within that).
3. Sample size (of child deaths): Despite the large study sample and the census of births (for both of which the researchers should be commended), there are approximately 250 deaths in the study sample, according to my back-of-the-envelope calculations. That's OK in the overall analysis. However, by the time we are dividing into different periods or looking at mechanisms and interactions with lots of other variables, you can very quickly get down to a handful of deaths in any given cell. I think that this might explain why some of the heterogeneity analysis produces very large impact estimates – for instance, practically eliminating deaths from birth complications in the neonatal period entirely, hence bringing Kenya to IMR levels seen in high-income countries. Even though the authors prefer the spatial IV specifications for a lot of their main findings, the heterogeneity of effects by the timing of cash using that analysis is relegated to Figure A.2, instead of main Figure 3, likely because the effect size (ES) during the birth month, at minus 57, is so large that it makes IMR negative in the treatment group for that subgroup. Even the effect size in Figure 3, at minus 40, is very large – likely rendering IMR to be (less than or) equal to zero…
The timing of the cash transfers and their (un)anticipated nature
Throughout the paper, I kept wondering about the exact timing of cash transfers and what people living in the study area knew and when. Specifically, I wasn’t sure how to think about,
· when people knew they would get cash transfers.
· when they knew which of the other nearby villages would get them.
· the 11-month period for three tranches of cash transfers, which is long enough that the transfer period can overlap with as many as three periods (e.g., in utero, birth month, and infancy).
The ambiguity emanating from these issues is important, in my mind, for the following reason. If the transfers were unanticipated and were given in a single tranche, then by focusing on HHs that randomly got their transfers within a small window of the birth month, we could rule out any selection (into childbearing) effects and be confident that there is some interaction effect between the timing of the payment and the likelihood of facility-based deliveries (and any knock-on effects from that on IMR, etc.).
But, as at least part of this story is not correct (recipients got their transfers in three tranches over a period of approximately one year), so it would be good for the authors to clarify how the reader should think about the timing of treatment, behavioral responses/anticipation effects of knowing ahead of time, targeting, etc. [Perhaps the authors could show effects on people who were already pregnant when they learned that they would be recipients?]. If we are going with the story that the reductions in IMR would have been much lower without spillovers, then the answers to these questions might help explaining the outsized role of spillover effects in the study findings – specifically how they came about…
What do we think of the story the paper is telling us?
IMR (and MMR) is a devastating problem in LMICs: too many preventable maternal and child deaths are due to complications in childbirth. The new study is implicitly suggesting that the problem is that (a) we don't have good facilities, (b) when we have them most people who need them can't get to them, and (c) even when they could, they don't because the service is too expensive. So, the narrative goes, if you were to get a bunch of things right (e.g., give money) to the right people (e.g., pregnant women) at exactly the right moment (e.g., just before delivery) in the right place (e.g., living near good hospitals), then you would succeed. Even then, you'd have to give this money to a lot more other people, this money would have to be so much larger than any government would consider giving in LMICs, and so on…
But isn't this just reinventing CCTs from a quarter century ago? If we gave people money to attend their recommended antenatal visits; inform them of the risks for complications during pregnancy, delivery, and immediately afterwards; and make hospital deliveries (and neonatal follow-ups) free for poor people (or even pay them to deliver at the hospital and for follow-up visits), would that not accomplish the same thing? After all, “targeting pregnant women from poor households,” (suggested by this paper in Section 4.5, ppg. 22) is exactly what many CCTs around the world have been doing for decades. Is all that distinguishes those earlier studies of CCTs from this one is that the former did not have the sample size to detect effects on infant mortality?
The study gives us good news. We need all the good news we can get these days, and we may not care to know exactly how it happened. But it’s still good to think about how good this piece of news really is: should we be ecstatic or cautiously optimistic?
Join the Conversation