Syndicate content

Blogs

Decomposing response error to improve consumption survey design

Jed Friedman's picture

Consumption or income, valued at prevailing market prices, is the workhorse metric of economic welfare – poverty is almost universally defined in these terms. In low- and middle-income countries these measures of household resource availability are typically assessed through household surveys. Yet the global diversity in survey approaches is vast, with little rigorous evidence concerning which particular approach yields the most accurate resource estimate. (Indeed there may be no one approach that best suits every context – more on this below.)

With this question in mind, Kathleen Beegle, Joachim DeWeerdt, John Gibson, and I conducted a survey measurement experiment in Tanzania that randomized common survey approaches to consumption measurement across a representative sample of households in Tanzania. Previous papers have explored the relative performance of the approaches in terms of mean consumption, inequality, poverty, and the prevalence of hunger (see these papers here, here, and here). Our new working paper seeks to push this data further to understand the nature of the reporting errors that underlie the mean estimates.

Weekly links April 22: development engineering, reporting context, the downside of good behavior, and more…

David McKenzie's picture
  • The inaugural issue of Development Engineering is now out (all issues are open access!). I’m delighted that my paper on attempting to use RFID to track small firm sales is in this first issue, along with a paper on how to randomize better in sequential randomized trials, a paper that proposes a “system [which] leverages smartphones, cellular based sensors, and cloud storage and computing to lower the entry barrier to impact evaluation”, a paper on biomass stoves, and one on rural electrification. Note also this from the editor’s introduction “we see major benefits from publishing studies that find weak or no impacts. In global development, there should be no silent failures; there is inherent value in learning from interventions that fail to achieve their intended impacts.”

Some theory on experimental design…with insight into those who run them

Markus Goldstein's picture
A nice new paper by Abhijit Banerjee, Sylvain Chassang, and Erik Snowberg brings theory to how we choose to do evaluations – with some interesting insights into those of us who do them.  It’s elegantly written, and full of interesting examples and thought experiments – well worth a read beyond the injustice I will do it here.  

From my inbox: Three enquiries on winsorizing, testing balance, and dealing with low take-up

David McKenzie's picture

I’ve been travelling the past week, and had several people contact me with questions about impact evaluation while away. I figured these might come up again, and so I’d put up the questions and answers here in case they are useful for others.
Question 1: Winsorizing – “do we do this on the whole sample, or do we do it within treatment and control, baseline and follow-up?”
Winsorizing is commonly used to deal with outliers, for example, you might set all data points above the 99th percentile equal to the 99th percentile. It is key here that you don’t use different cut-offs for treatment and control. For example, suppose you have a treatment for businesses that makes 4 percent of the treatment group grow their sales massively. If you winsorize separately at the 95th percentile of the treatment distribution for the treatment group and at the 95th percentile of the control distribution for the control groups, you might end up completely missing the treatment effect. I think it makes sense to do this with separate cutoffs by survey round to allow for seasonal effects and so you aren’t winsorizing more points from one round than another (which could be the case if you used the same global cutoffs for all rounds).

GiveDirectly just announced a basic income grant experiment. Here is how to make it better.

Berk Ozler's picture

In an article in Slate yesterday, co-founders of GiveDirectly announced that they will provide at least 6,000 people in Kenya with a basic income grant (BIG) for a period of 10-15 years, which will cost about $30 million. The proposal is scant in details at the moment, but this article in Vox suggests that dozens of villages will randomly be selected in an already selected region of Kenya for this exercise and everyone within will be given roughly a dollar a day per person for a decade.

Excuse me Mr. Can’t you see the children dying?

Berk Ozler's picture

In 1997, Peter Singer wrote about a dilemma he’d pose to his students about a drowning child in a pond on their way to class: “would they be willing to save the child at the cost of getting all wet, having to go back home to change, and missing the first period?” After getting the expected answer that they all would, he’d ask about a hypothetical child far away, and ways that the students could save lives elsewhere at “no great cost – and absolutely no danger – to themselves. This would lead to a discussion of a version of effective altruism two decades ago.

Friday Links: Frustrating research, Fryer's tome, sinister virus, and more...

Berk Ozler's picture

Poverty reduction through large asset transfers: a look at the long run

Markus Goldstein's picture
Last year, Banerjee and coauthors published a paper in Science that showed the striking impacts of poverty graduation programs in 6 countries after three years.   This week, we get a new paper from Bandiera and coauthors that revisits one of the models of this type of program they wrote about in 2013 and looks not only at a wide range of benefits, but also at what happens in the longer run.  

Teacher Turnover and Student Performance

Berk Ozler's picture

Selective teacher retention policies are both complex and controversial wherever they’re implemented. In Washington, DC, where I live and work, things are no different: pursuit of such policies arguably led to the early departures of a mayor and the chancellor of the public school system (DCPS). A January 2016 paper by Adnot, Dee, Katz, and Wyckoff evaluates the effect of teacher turnover under on student achievement under IMPACT, DCPS’ performance assessment and incentive system for its teachers – introduced in the 2009-10 school year (NBER WP version, gated, can be found here.

Pages