Published on Development Impact

Weekly links June 10: hardworking poor people, the practicalities of RCTs, what is clever in marketing experiments, marshmallow tests revisited, and more…

This page in:
  • From VoxEU – people in developing countries not only earn a lot less than those in developed countries, they also work longer hours and have less leisure – as a result “Adults in poor countries …are even less productive than we thought.”
  • All the Stata cheatsheets from Geocenter in one place
  • In Scientific American: does financial stress literally hurt?
  • In the Washington Post wonkblog, more evidence why the Marshmallow test is probably not measuring what you think it is – “this uncovers a broader problem with how we perceive the actions of people who live very different lives than we do. We brand certain actions and choices as mistakes, when they might simply be developmental adjustments necessary to cope with their environment. For those who don't worry about their next meal, because they never had to, choosing a marshmallow now instead of two marshmallows in a few minutes, all things equal, could only be the result of impulse-driven folly. For those who do have to worry about the next meal, passing up food now for the promise of food later is the misguided move.”
  • J-PAL now has almost all the chapters from the forthcoming Handbook of Field Experiments all up in one place. A fantastic set of papers, several of which I’ve linked to previously. New papers I hadn’t seen include:
    • Athey and Imbens on the Econometrics of Randomized Experiments: much of it is contained in the Imbens and Rubin book, but some of the newer content I want to understand more includes discussion (section 10.3.1) of testing for heterogeneity using data-driven approaches “without having to prespecify the form of the heterogeneity, and without having to worry about multiple testing” and also non-parametric estimation of heterogeneous treatment effects; as well as treatment effects with network interactions.
    • Glennerster on the practicalities of running RCTs: on the challenges of working with Governments: “Some governments have laws requiring them to treat citizens of equivalent need equally. When the Government of France wanted to test programs using randomized trials they first had to change the constitution to make this possible…Government procurement rules can also cause considerable delay. For example, if we decided that an intervention needs a leaflet to explain the study to participants, the government may require a competitive bid for the printing of the leaflet, leading to several months delay.” And on data transparency “But it is not clear that posting raw data is in fact more transparent. Most raw data requires so much work that it is impenetrable to anyone not involved in the study. Even if the raw data and the code to turn it into clean data is posted, the cleaning files will be so long and tedious that it is unlikely anyone will learn anything useful from them”
    • Hanna and Karlan on Designing Social Protection Programs:  useful discussion on why it might not be enough on targeting to simply use the same sample to compare alternative targeting criteria; and discussion of areas where we have very little evidence (unemployment insurance and pensions in developing countries).
    • Simester on Field experiments in marketing: one result is that discounts and extended payment terms can actually lower demand (by signaling lower quality); the difficulty of measuring advertising impacts “Despite a controlled randomized environment and a sample size of 1.6 million customers, they are only just able to establish a statistically significant effect.”;  why you might not want to be the first hit on google if you are a firm (if you are lower down, only the more motivated customers click through, and so if you have to pay per click-through, the cost-per-sale is lower if you are lower down). Also “Researchers are expected to also shed light on the mechanism that causes the effect …Insightful interactions are often viewed as the “clever” element of a study.”


David McKenzie

Lead Economist, Development Research Group, World Bank

Join the Conversation

The content of this field is kept private and will not be shown publicly
Remaining characters: 1000