Since Northern Uganda is very much in the news this week, I thought I would discuss an interesting paper by Chris Blattman, Nathan Fiala, and Sebastian Martinez which looks at the impact of a youth vocational training program in Northern Uganda (paper here and a short policy note here).
The program is a group based training and development grant. Groups of youth get together and submit an application for training, equipment and start-up capital. While there is a group managed committee, there is no oversight by the government once the money has been given to the group (the government does do an audit, including making sure the group exists before the funds are handed over). Funding is substantial, on average about $7,000 per group or $374 per member.
Luckily for Blattman and co., there were more eligible groups than there was money. A side note: I have grown skeptical of program implementers claims that there will be excess demand. One of these recent experiences was where a program where we were in fact giving away money, and there wasn’t even enough demand to equal the amount we had. More on this in a later post.
So, with this excess demand amongst qualified applicants, randomization is possible and indeed done in this case. This lets Blattman and co (who bring a nice mix of political science and economics skills to the table) look at outcomes in a range of dimensions.
First, did the kids just run off with the money? Apparently not – around 2/3 of the transfer was spent on vocational training fees and durable assets. And about 70% of the participants enrolled in some kind of vocational training course (top two: tailoring and carpentry). Assets stocks and acquisitions reported by participants go up – by 481% and 150% respectively, relative to the controls.
Now, the initial regressions show that the effects on assets are significantly lower for females. However, one thing this paper pays close attention to is the impacts across the distribution – throughout they use the inverse hyperbolic sine transformation, as well as quantile analysis to look at the robustness of their results. In this instance, the male-female gap does not appear to be robust (but the average findings hold).
The training and the capital seem to pay off. More of the treated folks are engaged in skilled or capital intensive enterprises, and earnings go up by on average $9 per week relative to the controls (again, an initial female difference is not robust to non-linear estimates). Now this $9 increase might seem small, but it’s a 45% bump relative to the earnings in the control group. Moreover, in order to put this further in perspective, Blattman and co. compute the rate of return on the project investment at 2.9% per month or 35% per annum (non-compounded). This compares favorably with the real prime lending rate (5%) and estimates of commercial lending to small firms (10-20%) but not microfinance (200% per annum) or moneylenders.
They also take a look at social outcomes, given that one of the goals of this program is to help these post-war communities get back on track (note that a lot of the communities within the program were hit by the civil war, but the three most hard-hit areas weren’t in this program). They find some, but not really large impacts on engagement in groups: those in the program had more group participation, were more likely to speak in meetings, and more likely to be a community mobilizer. Folks in the program were also more likely to report social support (4.7% more than the control group) and depression was lower – for males only.
When they look at aggressive behavior, things get really interesting, and somewhat puzzling. For men, there is a sharp drop in disputes with community leaders and the police. However, for women there is an increase in physical fights – the treatment group is twice as likely to report a physical fight vs the control. This makes them look a lot like the average man (around 5% of whom get into fights). Self reported hostile behavior also shows a similar pattern – measures decline for men and increase for women (including quarrelsomeness and threatening others).
So what’s going on? The short answer is, we don’t know. Blattman and co. argue that this increase in aggression may simply be a result of women engaging more in the market (after all, they don’t pass the level reported by males). However, they can’t trace this out from their data. But, the good news is that this is a midline report and we’ll see more on this, and other issues, from them in the next year or two.
Finally, Blattman and co. also do a nice job at looking for heterogenous effects. On one level, they look at a range of group characteristics at baseline to see how these change results. They use measures such as whether the group previously existed, quality of the group dynamic (based on questions posed to members), group size, proportion female, and a heterogeneity index. As it turns out, not many of these matter in important ways for a range of outcomes. One notable exception is the proportion of females – groups with a higher percent of females invest more in training, but with lower profit and wealth outcomes. Again, something worth explaining further in the next iteration of this work.
They also look at heterogeneous effects at the individual level. First, the initial working capital that folks have doesn’t seem to impact the magnitude of the treatment effects. Ability seems to lead to more training hours, but none of the other outcomes (investment, earnings, or wealth). Now, they chalk this up to a bad measure of entrepreneurship (they use education, literacy, digit recall, and physical and mental health) – and in one sense they must be right. After all, if any of us could measure entrepreneurship really, really well, then we’d be rich.
Join the Conversation