Thanks for the comments. On practical and technical grounds I don’t see the set of alternatives to any real policy yielding to isolated experiments. I don’t see an alternative to having theory (studied in whatever ways we can) filling in lots of gaps. For many policies (anything to do with changing prices, for example) studies of the RCT variety will not have a sufficient number of “arms” to study all relevant cross price effects (let alone the market failure in each of these related markets). Not enough power. And if we were to go for studies that would cover enough such alternatives, we’ll still need some structure (like demand systems, production possibility frontiers) to guide us over the, necessary, gaps in data. These could be built up derivative by derivative, product by product, country by country by individual experiments but all such policies based on them will be irrelevant before we’re half done.
The other big problem is that of the size of the relevant problem. Are we going to experiment with road systems (not the individual roads – unconnected roads are irrelevant: I know this because of… ummm… common derision of “roads to nowhere”? Or have people build in useless places just to avoid the endogenous placement problem), port location, power grids, sewer systems, mass transit? The big problems of development could be those of large public infrastructure. Our experiments are often with private goods since these are easily measured by household or factory surveys.
In order to make a problem tractable for experiments it is often conceived in ridiculously narrow terms. One experiment I like is the northern Ghana work of Karlan and Udry since it captured the conditions of farmers in broader terms than other studies of fertilizer use. When a fuller picture, involving the risk that farmers face in output markets, was included, the “irrationality” we imputed to them seemed to be attenuated by realizing they were solving a different problem than the researchers were. But that is an old insight – Lipton 1976? (72? 78?).
If experiments help us with containing air pollution (or global warming?) or measuring the labor market equilibrium of better trained workforce (and the externality of that training), that would be great. But I doubt they will.
Some things should be studied with experiments if that’s the right method for the right problem. But the problem should come first, the appropriate method to be matched to it.
Add new comment