Syndicate content

measurement

Issues of data collection and measurement

Berk Ozler's picture
About five years ago, soon after we started this blog, I wrote a blog post titled “Economists have experiments figured out. What’s next? (Hint: It’s Measurement)” Soon after the post, I had folks from IPA email me saying we should experiment with some important measurement issues, making use of IPA’s network of studies around the world.

What’s New in Measuring Subjective Expectations?

David McKenzie's picture

Last week I attended a workshop on Subjective Expectations at the New York Fed. There were 24 new papers on using subjective probabilities and subjective expectations in both developed and developing country settings. I thought I’d summarize some of the things I learned or that I thought most of interest to me or potentially our readers:

Subjective Expectations don’t provide a substitute for impact evaluation
I presented a new paper I have that is based on the large business plan competition I conducted an impact evaluation of in Nigeria.  Three years after applying for the program, I elicited expectations from the treatment group (competition winners) of what their businesses would be like had they not won, and from the control group of what their businesses would have been like had they won. The key question of interest is whether these individuals can form accurate counterfactuals. If they could, this would allow us a way to measure impacts of programs without control groups (just ask the treated for counterfactuals), and to derive individual-level treatment effects. Unfortunately the results show neither the treatment nor control group can form accurate counterfactuals. Both overestimate how important the program was for businesses: the treatment group thinks they would be doing worse off if they had lost than the control group actually is doing, while the control group thinks they would be doing much better than the treatment group is actually doing. In a dynamic environment, where businesses are changing rapidly, it doesn’t seem that subjective expectations can offer a substitute for impact evaluation counterfactuals.

Who in this household has the final say?

Markus Goldstein's picture
Who in the household has decision making power over various things (kids going to school, health seeking behavior of individual members) either alone or jointly with someone else in the household makes up a set of questions that often find their way into surveys (e.g. a version is included in most Demographic and Health Surveys).  An interesting new paper by Amber Peterman and coauthors takes a hard look at these questions and what they might, or might not, be telling us.    

A curated list of our postings on Measurement and Survey Design

David McKenzie's picture
This list is a companion to our curated list on technical topics. It puts together our posts on issues of measurement, survey design, sampling, survey checks, managing survey teams, reducing attrition, and all the behind-the-scenes work needed to get the data needed for impact evaluations.
Measurement

Improving the Granularity of Nighttime Lights Satellite Imagery: Guest Post by Alexei Abrahams

Popular data
Nighttime lights satellite imagery (DMSP-NTL) are now a popular data source among economists. In a sentence, these imagery encompass almost all inhabited areas of the globe, and record the average quantity of light observed at each pixel (nominal size ~1km2) across cloud-free nights for every year, 1992-2012. In under-developed or conflicted regions, where survey or census data at a fine level of spatial and temporal disaggregation are seldom available or reliable or comparable over space or time, NTL and other satellite imagery can be an excellent resource. Recent economics papers have used NTL to study growth of cities in sub-Saharan Africa (Storeygard (2015)), production activity in blockaded Palestinian towns of the West Bank (Abrahams (2015), van der Weide et al (2015)), and urban form in China (Baum-Snow & Turner (2015)) and India (Harari (2015)).

Can you measure flows over short periods? Aka why Justin Wolfers might (NOT) want to reconsider that parenting study

David McKenzie's picture
[EDIT: I POSTED TOO HASTILY HERE, SEE AN ADDENDUM BELOW WHERE I AGREE WITH JUSTIN AFTER ALL]
Today in the Upshot, Justin Wolfers heavily criticizes a recent study that has received lots of media attention claiming that child outcomes are barely correlated with the time that parents spend with their children. He writes:

9 pages or 66 pages? Questionnaire design’s impact on proxy-based poverty measurement

Talip Kilic's picture

This post is co-authored with Thomas Pave Sohnesen

Since 2011, we have struggled to reconcile the poverty trends from two complementary poverty monitoring sources in Malawi. From 2005 to 2009, the Welfare Monitoring Survey (WMS) was used to predict consumption and showed a solid decline in poverty. In contrast, the 2004/05 and 2010/11 rounds of the Integrated Household Survey (IHS) that measured consumption through recall-based modules showed no decline.
 
Today’s blog post is about a household survey experiment and our working paper, which can, at least partially, explain why complementary monitoring tools could provide different results. The results are also relevant for other tools that rely on vastly different instruments to measure the same outcomes.

Measuring Yields from Space

Florence Kondylis's picture

This post is co-authored with Marshall Burke.
One morning last August a number of economists, engineers, Silicon Valley players, donors, and policymakers met on the UC-Berkeley campus to discuss frontier topics in measuring development outcomes. The idea behind the event was not that economists could ask experts to create measurement tools they need, but instead that measurement scientists could tell economists about what was going on at the frontier of measuring development-related outcomes. Instead of waiting for pilot results, we decided to blog about some of these ideas and get inputs from Development Impact readers. In this series, we start with recent progress on measuring (“remote-sensing”) agricultural crop yields from space.

Pages