CSAE 2015: Impact Evaluation Round-up

I’ve just returned from the annual conference at the Center for the Study of African Economies, in Oxford, UK. More than 270 papers were presented in sessions ranging from education and health to political economy and monetary policy.

Lots of great work was presented; I recommend browsing the sessions for your areas of interest. Here is just a sample of some of the impact evaluation(ish) work plus some lab games, grouped by topic, to give you a sense of some of the Africa-related IE research coming down the line. I’ve indicated which papers are RCTs, in case you’re into those.

Agriculture and Land Tenure! 

• Providing commodity price info to rural farmers via text messages led to improved prices for treatment farmers. But – here’s the really cool thing – prices also went up for control farmers, potentially because traders had updated beliefs about what farmers knew [Hildebrandt et al.] #RCT
• Randomly pairing women farmers in rural Uganda and encouraging them to learn from each other increased cotton yields for all but the most productive at baseline, with positive spillovers to men [Vasilaky & Leonard] #RCT
• An “extension service provision and farmer training” program in Tanzania, implemented by BRAC, increase maize farmer productivity by 32% and incomes by 60% over two years [Okello] #RCT
• “Participating in contract farming [in Madagascar] reduces the duration of a household's hungry season by nine days.” Especially pronounced effect if you have lots of daughters. [Bellemare & Novak]
• Contract farming in Nigeria increases output by 80% in a quasi-experimental study [Awotide et al.]
• Land tenuring in Ethiopia increases household welfare [Ayalew]
• Providing feedback tools increased farmers’ demand for extension services in Rwanda with some interesting heterogeneous effects [Jones & Konylis] #RCT
• Comparing cash transfers to an index-based insurance product in Kenya reveals that both benefit participants, with similar benefits per average cost, but the insurance has much higher benefits per marginal cost. [Jensen et al.] #RCT (+ IV, but there’s an RCT in there)

Credit and savings!

• When offered a savings or a credit product in rural Pakistan, women took both, showing people don’t just want to defer (savings) or expedite (credit) consumption [Afzal et al.] #RCT
• Watching a Nollywood movie that encouraged responsible saving led to 5 percentage point more savings accounts being opened on the spot, but no effect on financial behavior 4 months later [Coville et al.] #RCT
• Commitment savings products may not be all that: 55 percent of clients default on a commitment savings contract in the Philippines. It seems the “chosen stakes were too low.” [John] #RCT (blog post)
• A program to promote savings (providing info on importance and existing instruments) doesn’t increase savings, but it shifts it to more formal instruments, especially for people who have experience theft in the past (25% at baseline) [Buehren] #RCT

Education! 

• Remember that conditional versus unconditional cash transfer program in Malawi from Baird et al.? Two years later after the program ended, the CCTs had enduring impacts for participants who were school dropouts at baseline, but the effects had fizzled for both CCTs and UCTs among girls who were in school at baseline [Baird et al.] #RCT
• Remember that program providing grants for vocational training and business start-ups in Uganda and increased business assets by 57% after 4 years from Blattman et al? It significantly increased education expenditures too. [Calderone] #RCT
• Experimental and social network data from Colombian schools show that CCTs had a positive direct effect on attendance, a positive endogenous peer effect (your friends come more because you come more), but a negative contextual effect (what happens are more people are treated). Net effect of the program? Zero. [Dieye et al.] #RCT
• School meals in Senegal increase enrollment, reduce dropout, and test scores for both French and math. Bonus points for a cost-effectiveness analysis. [Azomahou et al.] #RCT
• What’s the impact of British vs French colonization on current education? Using the split in Cameroon, looks like French colonization means lower secondary completion because of high repetition [Dupraz]
• The Indonesian 1998 economic crisis – ten years later – may have improved education for some children since they couldn’t find work at the time [Sharma]
• An unconditional child grant program in Zambia, targeted to households with children under five, increased enrollment for age 4-7 and also improved the transition to lower secondary school. Also reduced child labor for age 11-14. [Handa et al.] #RCT
• The South African child support grant (unconditional) increases school enrollment for teenagers [Eyal & Woolard]
• Creating school management committees in Burkina Faso increased people’s voluntary contributions to public goods by 8.5-24% [Sawada et al.]

Energy and Natural Resources!

• Major mineral discoveries (assumed to be exogenous) improve local living standards across Africa on average. No evidence of a resource curse. [Bhattacharyya et al.]
• Wait on that last one. Oil discoveries in Nigeria negatively affect communities in development of water, sanitation, electricity, etc. [Bruederle & Hodler]
• Running a mine in South Africa lowers crime, but closing it boosts it (a lot) [Axbard et al.]
• Low cost solar kits in Rwanda (good for lighting but can’t do both lighting and a mobile phone) decreased energy spending but probably still too expensive for unsubsidized uptake [Grimm et al.] #RCT

Firms!

• A tale of two business trainings for female entrepreneurs in Tanzania: In-class training alone had no impact. Enhanced training, with visits to the business premises, increased adoption of new practices. Revenues go up only for more experience entrepreneurs. [Gassier] #RCT
• Providing assistance for costless business registration in Malawi led to 75% of offerees getting registered. Take up not so high for tax registration. [Campos et al.] #RCT

Governance and Tax!

• Single-window service centers for social welfare schemes in south India increased applications, especially for non-free schemes [Berg et al] #RCT
• Electronic billing machines in Rwanda increased value-added tax revenues, despite relatively low use of the mahcines [Eissa & Zeitlin]

Health!

• People in Uganda with lower limbs disabilities receive medical equipment. Life satisfaction increases in short run but falls back to pre-treatment levels, even for those with ongoing increased mobility. Interesting implications for subjective welfare measures. [Appleton et al.] #RCT
• In the same disability treatment intervention in Uganda, but using a discontinuity in treatment, women who received treatment were 25% more likely to be in the labor force than those who didn’t [Bridges et al.]
• Free vaccines did not statistically increase vaccination rates in middle-income countries, since most of the kids would have been vaccinated anyway [Dykstra et al.]
• Poor households in Burkina Faso doubled take-up of micro-insurance in response to a 50% premium subsidy. Their probability of losing a day of work halves.  Lovely discontinuity. [Schleicher et al.]
• Removing health user fees in Zambia – tested using a synthetic control [more on that method here] – did not affect health seeking behaviors, even for the poorest. So, basically, a cash transfer. [Aurélia et al.]

Measurement!

• People report fewer assets (and hence more poverty) in a long questionnaire versus a short one [Kilic & Sohneson] #RCT (blog post)

Political Economy!

• Doubling police officer salaries in Ghana led to higher bribe amounts, estimated using data on bribes from around West Africa: “You think you can buy me off with that?!” [Foltz & Opoku-Agyemang]
• Partisan electoral monitors in Argentina (not in Africa, but on a relevant topic) cause a 1.7% to 7% increase in vote count of their party, quasi-experimental evidence suggests [Casas et al]

And, last but not least, some games!

• Kenyan participants received 100 shillings and could send 50 back and receive 500 two weeks later. They could send the 50 back today or in a week. Most chose today, so maybe waiting imposes a cost of keeping track. [Haushofer]
• Local customary judges in Ethiopia rule over the results of ultimatum games (more on those) played by villagers. When a costly appeal to a fix law is introduced, customary rulings come much close to formal law rulings (reduced bias in favor of men and the judge’s acquaintances) [Cecchi & Melesse]
• People in villages with formal health insurance in Uganda contribute less in a public goods game than those without it, but the result is driven by those who don’t have insurance in the insurance villages [Cecchi et al.]
• Sierra Leoneans play a trading game: Socially connected partners (family, friends, or colleagues) treat each other more fairly. [Ross & Voors]
• In a modified dictator game [more on those] played by students, civil servants, and private sector adults in urban Uganda, participants split money between themselves and a local charity. Civil servants give the least, private sector individuals give the most. [Lehrer & Porter]
• Public goods games (more on those) in post-flooding Pakistan reveal that “frequent experiences with mild floods” positively affects social capital whereas having been in a severe flood negative affects social capital. Don’t show the malevolent social planner these results! [Afzal et al.]

My favorite title of the conference was “Continued Existence of Cows Disproves Central Tenets of Capitalism?”

If I missed your impact evaluation, please share it in the comments.

[This isn’t a comprehensive list. I generated random numbers to examine only a sample of sessions, and I skipped papers that I couldn’t summarize after a bit of perusal.]

Update: One reader pointed out that I had mischaracterized the results of the Baird et al. paper above. That was an error, and I have corrected it.