- Does shaming people to pay taxes work? Yes according to an experiment in the U.S., but only if they don’t owe too much. (h/t @dinapomeranz)
- Chris Blattman offers his take on “Does Economics have an Africa problem?” – is it just me, or is is this whole debate a bit too Africa-centric? Economics has at least as much a Middle East problem, or Eastern Europe problem, or East Asia problem – in my view more if we compare the amount of research activity devoted to these other regions.
- Sana Rafiq discusses how behavioral biases affect our survey questions on the Let’s Talk Development blog, in the context of trying to replicate some of Sendhil Mullainathan’s scarcity work: when asking whether people would travel across town to get a bargain, “There is no guarantee that the product will still be there once I go across town. It’s very likely that the product is gone by the time I get there.” Of course! By assuming the availability of the product, we had let our own implicit biases, based on our mental models, influence the design of the question.”
Bruce Wydick on the Impact of giving away TOMS Shoes: He gives kudos to TOMS for being open for evaluation and being responsive to findings, but what caught my eye was this observation: "The bad news is that there is no evidence that the shoes exhibit any kind of life-changing impact,..."
I received this email from one of our readers:
“I don't know as much about list experiments as I'd like. Specifically, I have a question about administering them and some of the blocking procedures. I read a few of the pieces you recently blogged about and have an idea for one of my own; however, here's what I'd like to know: when you send your interviewers or researchers out into the field to administer a list experiment, how do you ensure that they are randomly administering the control and treatment groups? (This applies to a developing country as opposed to a survey administered over the phone.) “
This question of how to randomize questions (or treatments) on the spot in the field is of course a much more general one. Here’s my reply:
A common question of interest in evaluations is “which groups does the treatment work for best?” A standard way to address this is to look at heterogeneity in treatment effects with respect to baseline characteristics. However, there are often many such possible baseline characteristics to look at, and really the heterogeneity of interest may be with respect to outcomes in the absence of treatment. Consider two examples:
A: A vocational training program for the unemployed: we might want to know if the treatment helps more those who were likely to stay unemployed in the absence of an intervention compared to those who would have been likely to find a job anyway.
B: Smaller class sizes: we might want to know if the treatment helps more those students whose test scores would have been low in the absence of smaller classes, compared to those students who were likely to get high test scores anyway.
- On the IGC blog, Eliana La Ferrera summarizes different work on fighting poverty with soap operas
- A new repository for data from IPA/J-PAL RCTs. My questionnaires and datasets are in the World Bank’s open data library – and cross-linked from my webpage.
- Dave Evan’s post on systematic reviews last week has had a long series of comments. This week separate response blog posts by a 3ie team and by Langer, Haddaway and Land on the Africa Evidence Network
- Since we just changed to daylight savings time in the US – the LA Times rounds up a set of research results which look at the impacts of daylight savings changes including “Springing forward prompts people to waste time on the Internet”
- IPA/J-PAL policy bulletin summarizing 7 microcredit RCTs “where credit is due” – very nice set of Tables and Figures that summarize the study features and results
This post is co-authored with Thomas Pave Sohnesen
Since 2011, we have struggled to reconcile the poverty trends from two complementary poverty monitoring sources in Malawi. From 2005 to 2009, the Welfare Monitoring Survey (WMS) was used to predict consumption and showed a solid decline in poverty. In contrast, the 2004/05 and 2010/11 rounds of the Integrated Household Survey (IHS) that measured consumption through recall-based modules showed no decline.
Today’s blog post is about a household survey experiment and our working paper, which can, at least partially, explain why complementary monitoring tools could provide different results. The results are also relevant for other tools that rely on vastly different instruments to measure the same outcomes.
- Chris Blattman rediscovers one of my favorite blog posts – on managers vs makers and how one meeting can kill your whole day
- Eval Blog’s 10 predictions for the future of evaluation “Most evaluations will be internal”, “Evaluation reports will become obsolete due to real-time data” and more…
- Vox on PLOS One’s new section for negative studies: a collection of negative, null, and inconclusive studies title “missing pieces” including a failure to replicate the idea that self-control gets depleted
Consumption or income, valued at prevailing market prices, is the workhorse metric of human welfare in economic analysis; poverty is almost universally defined in these terms, and the growth of national economies measured as such. Yet for almost as long as economic analysis has utilized these measures, various shortcomings have been noted in the ability of these constructs to comprehensively capture welfare. One example – these measures can’t fully account for access to non-market goods. More famously, with Amartya Sen’s emphasis on human functionings and capabilities, these measures may not fully capture an individual’s ability to achieve and exhibit agency.
In part inspired by this view that people intrinsically value capabilities and functionings as opposed to money-metric measures per se, a burgeoning sub-field of poverty research has proposed various measures of subjective, or self-reported, well-being (SWB). SWB is widely seen as multi-dimensional and unable to be captured in only one question. Hence there are numerous approaches to the measure of SWB, most notably combinations of evaluative/cognitive approaches, such those that inquire about life satisfaction, and hedonic/affective approaches such as those asking about happiness.
I think it’s uncontroversial if I claim that the field of economics is of mixed minds about the usefulness of SWB: these measures hold some promise for comprehensive welfare assessment yet there are various interpretive challenges. I’ve blogged about some of these challenges in the past. Most concerning is the worry that salient characteristics such as gender and education, which naturally vary in any population, influence how SWB questions are understood and reported, thus complicating cross-group comparisons. Now two recent papers have made advances in the field and, taken together, highlight both the pitfalls and the promise of SWB.