Syndicate content

Blogs

Endogenous stratification: the surprisingly easy way to bias your heterogeneous treatment effect results and what you should do instead

David McKenzie's picture

A common question of interest in evaluations is “which groups does the treatment work for best?” A standard way to address this is to look at heterogeneity in treatment effects with respect to baseline characteristics. However, there are often many such possible baseline characteristics to look at, and really the heterogeneity of interest may be with respect to outcomes in the absence of treatment. Consider two examples:
A: A vocational training program for the unemployed: we might want to know if the treatment helps more those who were likely to stay unemployed in the absence of an intervention compared to those who would have been likely to find a job anyway.
B: Smaller class sizes: we might want to know if the treatment helps more those students whose test scores would have been low in the absence of smaller classes, compared to those students who were likely to get high test scores anyway.

Weekly links March 13: Soap Operas, New Data Access, Daylight Saving and Goofing Off, and more…

David McKenzie's picture

9 pages or 66 pages? Questionnaire design’s impact on proxy-based poverty measurement

Talip Kilic's picture

This post is co-authored with Thomas Pave Sohnesen

Since 2011, we have struggled to reconcile the poverty trends from two complementary poverty monitoring sources in Malawi. From 2005 to 2009, the Welfare Monitoring Survey (WMS) was used to predict consumption and showed a solid decline in poverty. In contrast, the 2004/05 and 2010/11 rounds of the Integrated Household Survey (IHS) that measured consumption through recall-based modules showed no decline.
 
Today’s blog post is about a household survey experiment and our working paper, which can, at least partially, explain why complementary monitoring tools could provide different results. The results are also relevant for other tools that rely on vastly different instruments to measure the same outcomes.

Weekly links March 6: The future of evaluation, publishing negative/null results, Science publishes a non-experimental study, and more…

David McKenzie's picture

New developments with the pitfalls and the promise of subjective welfare

Jed Friedman's picture

Consumption or income, valued at prevailing market prices, is the workhorse metric of human welfare in economic analysis; poverty is almost universally defined in these terms, and the growth of national economies measured as such. Yet for almost as long as economic analysis has utilized these measures, various shortcomings have been noted in the ability of these constructs to comprehensively capture welfare. One example – these measures can’t fully account for access to non-market goods. More famously, with Amartya Sen’s emphasis on human functionings and capabilities, these measures may not fully capture an individual’s ability to achieve and exhibit agency.

In part inspired by this view that people intrinsically value capabilities and functionings as opposed to money-metric measures per se, a burgeoning sub-field of poverty research has proposed various measures of subjective, or self-reported, well-being (SWB). SWB is widely seen as multi-dimensional and unable to be captured in only one question. Hence there are numerous approaches to the measure of SWB, most notably combinations of evaluative/cognitive approaches, such those that inquire about life satisfaction, and hedonic/affective approaches such as those asking about happiness.
 
I think it’s uncontroversial if I claim that the field of economics is of mixed minds about the usefulness of SWB: these measures hold some promise for comprehensive welfare assessment yet there are various interpretive challenges. I’ve blogged about some of these challenges in the past. Most concerning is the worry that salient characteristics such as gender and education, which naturally vary in any population, influence how SWB questions are understood and reported, thus complicating cross-group comparisons. Now two recent papers have made advances in the field and, taken together, highlight both the pitfalls and the promise of SWB.

227 studies later, what actually works to improve learning in developing countries?

David Evans's picture
Yesterday we talked about some of the limitations in systematic reviews of educational research, and how many of the reviews have – on the face of them – varying recommendations. The main recommendations as to what works (principally drawn from the abstracts and introductions) are in the figure below.
 

Blog links February 27th: What counts as a nudge, being efficient, debiasing, and more…

David McKenzie's picture
  • How to be efficient – excellent advice from Dan Ariely In particular I liked “A calendar should be a record of anything that needs to get done — not merely of interruptions like meetings and calls.” and “frequent email checks can temporarily lower your intelligence more than being stoned”

Why is Difference-in-Difference Estimation Still so Popular in Experimental Analysis?

Berk Ozler's picture
David McKenzie pops out from under many empirical questions that come up in my research projects, which has not yet ceased to be surprising every time it happens, despite his prolific production. The last time it happened was a teachable moment for me, so I thought I’d share it in a short post that fits nicely under our “Tools of the Trade” tag.

Blog links February 20: understandability, the replication debate continues, thoughts on the “Africa problem in economics”, and more…

David McKenzie's picture
  • A third paper in 3ie’s internal replication series is now out – along with a response from the authors (Stefan Dercon and co-authors). The author’s response is interesting for some of the issues with such replication exercises that it raises “At the outset of this exercise, we were enthusiastic, but possibly naive participants. At its end, we find it hard to shake the feeling that an activity that began as one narrowly focused on pure replication morphed – once our original findings were confirmed (save for a very minor programming error that we willingly confess to) - into a 14 month effort to find an alternative method/structure of researching the problem that would yield different results.” (See also Berk’s posts on the previous replications).
  • On the Let’s Talk Development blog, Emanuela Galasso reflects on the Chile Solidario program and how social programs can move from social protection to productive inclusion.
  • From Cornell’s Economics that really matters blog – conducting fieldwork in a conflict zone in Mexico.

Pages