Syndicate content

David McKenzie's blog

Weekly links June 3: Small begets big, expert predictions, process evaluation, measurement, and more…

David McKenzie's picture

Book Review: Grit – Takeaways for Development Economists and Parents

David McKenzie's picture

Angela Duckworth’s new book Grit: The Power of Passion and Perseverance has been launched with great fanfare, reaching number two on the NY Times Nonfiction bestseller list. She recently gave a very polished and smooth book launch talk to a packed audience at the World Bank, and is working with World Bank colleagues on improving grit in classrooms in Macedonia. Billed as giving “the secret to outstanding achievement” I was interested in reading the book as both a researcher and a parent. I thought I’d continue my book reviews series with some thoughts on the book.

Weekly links May 20: AEA P&P Special Edition

David McKenzie's picture
The latest AEA papers and proceedings has a number of interesting papers:
  • In the Richard T. Ely lecture, John Campbell discusses the challenge of consumer financial regulation – he distinguishes 5 dimensions of financial ignorance many households exhibit: 1) ignorance of even the most basic financial concepts (financial illiteracy); 2) ignorance of contract terms (such as not knowing about the fees build into credit cards or when mortgage interest rates can change); 3) ignorance of financial history – relying too much on own experiences and the recent past; 4) ignorance of self- a lot of financially illiterate people are over-confident about their abilities; and 5) ignorance of incentives, strategy and equilibrium – failure to take account of incentives faced by other parties to transactions.  Given these problems, and the limits of financial education and disclosure requirements to fix them, he discusses what financial regulation is needed: “consumer financial regulation must confront the trade-off between the benefits of intervention to behavioral agents, and the costs to rational agents….the task for economists is to confront this trade-off explicitly”

What’s New in Measuring Subjective Expectations?

David McKenzie's picture

Last week I attended a workshop on Subjective Expectations at the New York Fed. There were 24 new papers on using subjective probabilities and subjective expectations in both developed and developing country settings. I thought I’d summarize some of the things I learned or that I thought most of interest to me or potentially our readers:

Subjective Expectations don’t provide a substitute for impact evaluation
I presented a new paper I have that is based on the large business plan competition I conducted an impact evaluation of in Nigeria.  Three years after applying for the program, I elicited expectations from the treatment group (competition winners) of what their businesses would be like had they not won, and from the control group of what their businesses would have been like had they won. The key question of interest is whether these individuals can form accurate counterfactuals. If they could, this would allow us a way to measure impacts of programs without control groups (just ask the treated for counterfactuals), and to derive individual-level treatment effects. Unfortunately the results show neither the treatment nor control group can form accurate counterfactuals. Both overestimate how important the program was for businesses: the treatment group thinks they would be doing worse off if they had lost than the control group actually is doing, while the control group thinks they would be doing much better than the treatment group is actually doing. In a dynamic environment, where businesses are changing rapidly, it doesn’t seem that subjective expectations can offer a substitute for impact evaluation counterfactuals.

Weekly links May 6: expensive African cities, lotteries for housing, placebo effects, and more…

David McKenzie's picture
  • I liked the recent Planet Money podcast #698 (a long way home) – there is an interesting discussion of why a lottery is held for access to a housing assistance program in Connecticut, and how they ended up with a lottery rather than other systems of allocating resources – and a great quote about the mishmash of anti-poverty programs in the U.S. which, paraphrasing, is basically “it is not like Congress ever sat down and said what is the best use of the money we set aside to fight poverty” but rather how many different programs have come up over time, all with their own rules and constituencies.
  • The latest Journal of Economic Perspectives has a symposium on inequality beyond income (US focused) and a paper on the billion prices project that I linked to a blog post on last week
  • Should policy seek to promote small firms or large ones in Africa? Frances Teal on the CSAE blog: “Policy rhetoric focuses on the problems faced by small firms. Data from Ghana over the period for which we have it suggests that it is large firms that face the problems. Unloved possibly because they are not seen as beautiful they are vital for the output of the sector. Policy, not for the first time in Africa, seems to be focused on completely the wrong problem.”
  • The Los Angeles Review of Books has a longish discussion on placebo effects in reviewing an anthology on placebos, and how they really don’t work as much of the time in medicine as many people think, and how the term might be over-used in social sciences.
  • New in the working paper series: Is living in African cities expensive? Using data from the “2011 round of the International Comparison Program. Readjusting the calculated price levels from national to urban levels, the analysis indicates that African cities are relatively more expensive, despite having lower income levels. The price levels of goods and services consumed by households are up to 31percent higher in Sub-Saharan Africa than in other low- and middle-income countries, relative to their income levels. Food and non-alcoholic beverages are especially expensive, with price levels around 35 percent higher than in other countries.”

An addendum to pre-analysis plans: Pre-specifying when you won’t use data collected

David McKenzie's picture

Researchers put a lot of effort into developing survey questionnaires designed to measure key outcomes of interest for their impact evaluations. But every now and then, despite efforts piloting and fine-tuning surveys, some of the questions end up “not working”.  The result is data that are so noisy and/or missing for so many observations that you may not want to use them in the final analysis. Just as pre-analysis plans have a role in specifying in advance what variables you will use to test which hypotheses, perhaps we also want to specify some rules in advance for when we won’t use the data we’ve collected. This post is a first attempt at doing so.

Weekly links April 29: claiming your failures, what a billion prices tells us, the demand for health products, and more…

David McKenzie's picture

Weekly links April 22: development engineering, reporting context, the downside of good behavior, and more…

David McKenzie's picture
  • The inaugural issue of Development Engineering is now out (all issues are open access!). I’m delighted that my paper on attempting to use RFID to track small firm sales is in this first issue, along with a paper on how to randomize better in sequential randomized trials, a paper that proposes a “system [which] leverages smartphones, cellular based sensors, and cloud storage and computing to lower the entry barrier to impact evaluation”, a paper on biomass stoves, and one on rural electrification. Note also this from the editor’s introduction “we see major benefits from publishing studies that find weak or no impacts. In global development, there should be no silent failures; there is inherent value in learning from interventions that fail to achieve their intended impacts.”

From my inbox: Three enquiries on winsorizing, testing balance, and dealing with low take-up

David McKenzie's picture

I’ve been travelling the past week, and had several people contact me with questions about impact evaluation while away. I figured these might come up again, and so I’d put up the questions and answers here in case they are useful for others.
Question 1: Winsorizing – “do we do this on the whole sample, or do we do it within treatment and control, baseline and follow-up?”
Winsorizing is commonly used to deal with outliers, for example, you might set all data points above the 99th percentile equal to the 99th percentile. It is key here that you don’t use different cut-offs for treatment and control. For example, suppose you have a treatment for businesses that makes 4 percent of the treatment group grow their sales massively. If you winsorize separately at the 95th percentile of the treatment distribution for the treatment group and at the 95th percentile of the control distribution for the control groups, you might end up completely missing the treatment effect. I think it makes sense to do this with separate cutoffs by survey round to allow for seasonal effects and so you aren’t winsorizing more points from one round than another (which could be the case if you used the same global cutoffs for all rounds).

Get more farmers off their farms

David McKenzie's picture
Justin Wolfers had a nice piece in the Upshot about new work on how growing up in a bad neighborhood has long-term negative consequences for kids. The key point of the new work is that the benefits of moving from bad neighborhoods may be particularly high for kids whose parents won’t voluntarily move, but only move because their public housing is demolished.