Syndicate content

David McKenzie's blog

Can you measure flows over short periods? Aka why Justin Wolfers might (NOT) want to reconsider that parenting study

David McKenzie's picture
[EDIT: I POSTED TOO HASTILY HERE, SEE AN ADDENDUM BELOW WHERE I AGREE WITH JUSTIN AFTER ALL]
Today in the Upshot, Justin Wolfers heavily criticizes a recent study that has received lots of media attention claiming that child outcomes are barely correlated with the time that parents spend with their children. He writes:

Tips for Randomization in the Wild: Adding a Waitlist

David McKenzie's picture
This is a relatively small point, but one that has come up several times in conversations in the last few months, so I thought it worth noting here.
Context: you are randomly selecting people for some program such as a training program, transfer program, etc. in which you expect less than 100% take-up of the treatment from those assigned to treatment. You are relying on an oversubscription design, in which more people apply for the course/program than you have slots.

Blog links March 27: grants spur innovation, shaming tax scofflaws, the risks of piloting, why it is hard to work with academics, and more…

David McKenzie's picture
  • Does shaming people to pay taxes work? Yes according to an experiment in the U.S., but only if they don’t owe too much. (h/t @dinapomeranz)
  • Chris Blattman offers his take on “Does Economics have an Africa problem?” – is it just me, or is is this whole debate a bit too Africa-centric? Economics has at least as much a Middle East problem, or Eastern Europe problem, or East Asia problem – in my view more if we compare the amount of research activity devoted to these other regions.
  • Sana Rafiq discusses how behavioral biases affect our survey questions on the Let’s Talk Development blog, in the context of trying to replicate some of Sendhil Mullainathan’s scarcity work: when asking whether people would travel across town to get a bargain, “There is no guarantee that the product will still be there once I go across town. It’s very likely that the product is gone by the time I get there.” Of course! By assuming the availability of the product, we had let our own implicit biases, based on our mental models, influence the design of the question.”

From my email correspondence: how to randomize in the field

David McKenzie's picture

I received this email from one of our readers:
“I don't know as much about list experiments as I'd like.  Specifically, I have a question about administering them and some of the blocking procedures.  I read a few of the pieces you recently blogged about and have an idea for one of my own; however, here's what I'd like to know: when you send your interviewers or researchers out into the field to administer a list experiment, how do you ensure that they are randomly administering the control and treatment groups? (This applies to a developing country as opposed to a survey administered over the phone.) “
This question of how to randomize questions (or treatments) on the spot in the field is of course a much more general one. Here’s my reply:

Endogenous stratification: the surprisingly easy way to bias your heterogeneous treatment effect results and what you should do instead

David McKenzie's picture

A common question of interest in evaluations is “which groups does the treatment work for best?” A standard way to address this is to look at heterogeneity in treatment effects with respect to baseline characteristics. However, there are often many such possible baseline characteristics to look at, and really the heterogeneity of interest may be with respect to outcomes in the absence of treatment. Consider two examples:
A: A vocational training program for the unemployed: we might want to know if the treatment helps more those who were likely to stay unemployed in the absence of an intervention compared to those who would have been likely to find a job anyway.
B: Smaller class sizes: we might want to know if the treatment helps more those students whose test scores would have been low in the absence of smaller classes, compared to those students who were likely to get high test scores anyway.

Weekly links March 13: Soap Operas, New Data Access, Daylight Saving and Goofing Off, and more…

David McKenzie's picture

Weekly links March 6: The future of evaluation, publishing negative/null results, Science publishes a non-experimental study, and more…

David McKenzie's picture

Blog links February 27th: What counts as a nudge, being efficient, debiasing, and more…

David McKenzie's picture
  • How to be efficient – excellent advice from Dan Ariely In particular I liked “A calendar should be a record of anything that needs to get done — not merely of interruptions like meetings and calls.” and “frequent email checks can temporarily lower your intelligence more than being stoned”

Blog links February 20: understandability, the replication debate continues, thoughts on the “Africa problem in economics”, and more…

David McKenzie's picture
  • A third paper in 3ie’s internal replication series is now out – along with a response from the authors (Stefan Dercon and co-authors). The author’s response is interesting for some of the issues with such replication exercises that it raises “At the outset of this exercise, we were enthusiastic, but possibly naive participants. At its end, we find it hard to shake the feeling that an activity that began as one narrowly focused on pure replication morphed – once our original findings were confirmed (save for a very minor programming error that we willingly confess to) - into a 14 month effort to find an alternative method/structure of researching the problem that would yield different results.” (See also Berk’s posts on the previous replications).
  • On the Let’s Talk Development blog, Emanuela Galasso reflects on the Chile Solidario program and how social programs can move from social protection to productive inclusion.
  • From Cornell’s Economics that really matters blog – conducting fieldwork in a conflict zone in Mexico.

Evaluating an Argentine regional tourism policy using synthetic controls: tan linda que enamora?

David McKenzie's picture
In 2003, the Argentine province of Salta launched a new tourism development policy with the explicit objective of boosting regional development. This included improving tourism and transport infrastructure, restoring historical and cultural heritage areas, tax credits for the construction and remodeling of hotels, and a major promotion campaign at the national and international levels.

Pages