Context: you are randomly selecting people for some program such as a training program, transfer program, etc. in which you expect less than 100% take-up of the treatment from those assigned to treatment. You are relying on an oversubscription design, in which more people apply for the course/program than you have slots.
- Does shaming people to pay taxes work? Yes according to an experiment in the U.S., but only if they don’t owe too much. (h/t @dinapomeranz)
- Chris Blattman offers his take on “Does Economics have an Africa problem?” – is it just me, or is is this whole debate a bit too Africa-centric? Economics has at least as much a Middle East problem, or Eastern Europe problem, or East Asia problem – in my view more if we compare the amount of research activity devoted to these other regions.
- Sana Rafiq discusses how behavioral biases affect our survey questions on the Let’s Talk Development blog, in the context of trying to replicate some of Sendhil Mullainathan’s scarcity work: when asking whether people would travel across town to get a bargain, “There is no guarantee that the product will still be there once I go across town. It’s very likely that the product is gone by the time I get there.” Of course! By assuming the availability of the product, we had let our own implicit biases, based on our mental models, influence the design of the question.”
Bruce Wydick on the Impact of giving away TOMS Shoes: He gives kudos to TOMS for being open for evaluation and being responsive to findings, but what caught my eye was this observation: "The bad news is that there is no evidence that the shoes exhibit any kind of life-changing impact,..."
I received this email from one of our readers:
“I don't know as much about list experiments as I'd like. Specifically, I have a question about administering them and some of the blocking procedures. I read a few of the pieces you recently blogged about and have an idea for one of my own; however, here's what I'd like to know: when you send your interviewers or researchers out into the field to administer a list experiment, how do you ensure that they are randomly administering the control and treatment groups? (This applies to a developing country as opposed to a survey administered over the phone.) “
This question of how to randomize questions (or treatments) on the spot in the field is of course a much more general one. Here’s my reply:
A common question of interest in evaluations is “which groups does the treatment work for best?” A standard way to address this is to look at heterogeneity in treatment effects with respect to baseline characteristics. However, there are often many such possible baseline characteristics to look at, and really the heterogeneity of interest may be with respect to outcomes in the absence of treatment. Consider two examples:
A: A vocational training program for the unemployed: we might want to know if the treatment helps more those who were likely to stay unemployed in the absence of an intervention compared to those who would have been likely to find a job anyway.
B: Smaller class sizes: we might want to know if the treatment helps more those students whose test scores would have been low in the absence of smaller classes, compared to those students who were likely to get high test scores anyway.
- On the IGC blog, Eliana La Ferrera summarizes different work on fighting poverty with soap operas
- A new repository for data from IPA/J-PAL RCTs. My questionnaires and datasets are in the World Bank’s open data library – and cross-linked from my webpage.
- Dave Evan’s post on systematic reviews last week has had a long series of comments. This week separate response blog posts by a 3ie team and by Langer, Haddaway and Land on the Africa Evidence Network
- Since we just changed to daylight savings time in the US – the LA Times rounds up a set of research results which look at the impacts of daylight savings changes including “Springing forward prompts people to waste time on the Internet”
- IPA/J-PAL policy bulletin summarizing 7 microcredit RCTs “where credit is due” – very nice set of Tables and Figures that summarize the study features and results
This post is co-authored with Thomas Pave Sohnesen
Since 2011, we have struggled to reconcile the poverty trends from two complementary poverty monitoring sources in Malawi. From 2005 to 2009, the Welfare Monitoring Survey (WMS) was used to predict consumption and showed a solid decline in poverty. In contrast, the 2004/05 and 2010/11 rounds of the Integrated Household Survey (IHS) that measured consumption through recall-based modules showed no decline.
Today’s blog post is about a household survey experiment and our working paper, which can, at least partially, explain why complementary monitoring tools could provide different results. The results are also relevant for other tools that rely on vastly different instruments to measure the same outcomes.