I just finished reading Economic Gangsters, an excellent little book that summarizes in popular form a lot of recent evaluation work on corruption and violence. (See earlier posts here and here.) Although the book covers a hodgepodge of topics, from witchcraft in Tanzania to smuggling in China, it is held together by a particular approach to international development - namely, a call for randomized evaluations or one of its close kin to assess the utility of development interventions. Authors Raymond Fisman and Edward Miguel put it rather bluntly when discussing the particular case of the Millennium Village Project:
We genuinely hope that the Sauri [millennium village] model proves to be the great innovation that solves the problem of global poverty...But a serious program evaluation is needed to understand how and why its successes did (or did not) take place. Otherwise, we'll have learned little, and Sauri and the other Millennium Villages will likely join the long list of well-intentioned but ultimately inconclusive (and quickly forgotten) attempts to make poverty history.
I agree with their critique, but I wonder in turn what the limits are to their approach when looking at the particular issue of corruption. As Fisman and Miguel point out in the book, randomized evaluations have been invaluable in judging the merits of certain types of interventions. They provide examples of two interventions in Busia, a district in Kenya. One intervention provided deworming pills for children, while another supplied textbooks to classrooms. Since both interventions were randomly assigned, researchers were able to figure out after the fact that the deworming pills did a good job of boosting school attendance, while the additional textbooks had at best a negligible effect. Fisman and Miguel maintain that a similar approach can be used to learn how to fight corruption.
To support this point of view, the authors point to a randomized evaluation in Indonesia that tried to look at different ways to fight corruption in the building of roads. A lot of money tends to be siphoned off, as contractors use cheap materials or otherwise fail to deliver on their promises. The evaluation found that community town hall meetings did little to reduce corruption, while outside audits had a pretty substantial anti-corruption effect. I don't deny that this is a useful piece of information to know. But can this really be compared to the evaluations of deworming and textbooks in Kenya?
Evaluations of whatever kind are used to discover a relationship between some variables of interest that existed in the past. The results are used to justify future interventions because the assumption is that the underlying relationship still exists. In the case of deworming, I don't see anything controversial about this assumption. It seems pretty likely that if parasites were the main culprit behind low levels of school attendance a year ago, then they probably will continue to be a year from now. The same cannot be said of corruption, though.
By it's very nature, corruption is a dynamic process. Corrupt government officials are not stupid - in fact, they are usually ingenious, and they learn over time. Put in place external audits over road contracts? It may help at first at reducing corruption, but over time the contractors will figure out a way to bribe the auditors. In other words, the underlying relationship between the variables of interest disappears, and we're back to square one.
None of this is to say that randomized evaluations shouldn't be used to assess anti-corruption interventions, nor that Fisman and Miguel are ignorant of the issues I raise here. In fact, toward the end of the book, they point out that
"[r]andomized evaluations are not a panacea, and some questions will probably never be amenable to these evaluation methods." However, they never really address the dynamic nature of corruption, as compared to many other types of development problems. How long should we evaluate the outcomes of an anti-corruption initiative? How do we evaluate anti-corruption initiatives if they must adapt over time to ever-changing corrupt officials? For how long can we trust the results of a given evaluation, and how 'portable' is it, i.e. to what extent can we rely on the results of the Indonesian anti-corruption in other countries? Hopefully, these are topics we'll be hearing more about from Fisman and Miguel in the future.
Join the Conversation