Syndicate content

The perfectionists versus the reductionists

Markus Goldstein's picture

coauthored with Jishnu Das

Women perform 66 percent of the world’s work, and produce 50 percent of the food, yet earn only 10 percent of the income…. 

--Former President Bill Clinton addressing the annual meeting of the Clinton Global Initiative (September 2009)

Impressive, heart-wrenching, charity-inducing, get off your sofa and go do something heartbreaking.

But Wrong.

It’s a problem isn’t it? Almost all those twitter-sized 140 character, pull-all-the-right-heart-strings messages (& yes, the “oh-we-did-so-great” messages like David gently took aim at) are usually wrong. Or at least don’t stand up to detailed scrutiny without many, many caveats.

Working on the gender World Development Report, here are some of our favorite ones:

Women make up nearly 70 percent of the world's poor and 65 percent of the world's illiterate. (International Labour Organisation 1996)

Women produce half the world's food, but own only one percent of its farmland (CARE, “Women’s Day Facts”)

Increasing gender equality by ____________ would increase growth by ___ %.

We specially love this last format, because you can put in anything in the first and second blanks, and the statement would almost surely be wrong (the Gender WDR has a discussion (box 0.1) on why looking at the impact of gender equality on country-level growth is just a bad empirical idea. Period.) And people actually have put whatever they want in the blanks including (but by no means limited to): education, teenage pregnancy, employment, land ownership, women’s wages, voting rights and voice. This is not because gender equality has no impact on economic outcomes; the Gender WDR copiously documents it does. But it does reflect a growing consensus among economists that, apart from “institutions”, we really don’t know how to spur growth in any given country at a given time.

But before you think that we are against the pitching of such erroneous or deeply simplistic messages (that are usually wrong), think again. We aren’t.

The simple fact of the matter is that it comes down to the power of rhetoric. You need facts that would fit in a tweet (the fact that most of these messages predate Twitter suggests to us that the 140 character limit is endogenous to what big speech writers knew anyway). They’re simple, they’re dramatic, and they just might get you off of the couch. 

Contrast this with Markus’s attempted sales job on land titling reform in Rwanda, where his pitch on the results are: “Land titling reform in Rwanda increased female land ownership and tripled the proportion of women who invested in their land through soil conservation. But it actually lowered ownership rights for women who aren’t officially married and these results are for four pilot areas in Rwanda.  As the ultimate sales pitch, Markus adds helpfully for the readers: “What we can say for sure is that there is no evidence of a (short-term) significant sell-off of land….It will be interesting to see what happens as this program goes nationwide.

The blog received 0 comments (nada, zilch, nothing) and in Google’s blogsearch, we were unable to find any other blog that linked to this one. As yet another blog sinks into oblivion, it is worth recalling Banerjee and Duflo’s discussion in the beginning of “Poor Economics”: People respond more when given emotional triggers tied to simple problems rather than complex representations of worldwide issues. If the aim is to educate people and get them to either think of something new or do something, in a fast-paced busy world you have to catch their attention. And the only way to do that is be dramatic, pithy and damn the consequences.

People who want to get the messages out then have a tough job. First, they have to deal with perfectionist researchers who always want to contextualize and caveat their findings. Can you imagine every finding from an instrumental variable paper with the caveat: “Of course, this is only the Local Average Treatment Effect, and we really don’t know what the average treatment effect is, which depends on how essential heterogeneity plays out in this particular population”. Doesn’t really work in trying to reach a broad audience, does it? Even with RCT’s that are easier to explain, any researcher will always want to ensure that the audience understands the particular context within which the experiment was run (the issue of external validity).

Second, they have little to go with. On the one hand, the literature is increasingly giving up on the ability to make “big” statements linking any specific country investments to growth. On the other, the microeconomic evidence is very geographically concentrated in OECD countries, and particularly the U.S. For those working on low-income countries, like at The World Bank, UN or ILO, the paucity of academic work outside the OECD is just astounding. A recent working paper by Jishnu and coauthors, finds that over a 20-year span dating from 1985 to 2004, the top 5 economics journals published 39 papers on India, 65 papers on China, and 34 papers on all of Sub-Saharan Africa. In contrast, they published 2,383 papers on the United States. It’s not just the top-5 journals; over this period in the top 202 economics journals there was 1 paper on Chad, 2 on Guinea Bissau and 20 on Niger. At this rate, it is basically going to be impossible to say something that resonates for many countries in a catchy, pithy and correct fashion.

So what should we do? We have a suggestion, which comes from observing election practices in the U.S. As we all know, candidates go off the deep-end many times and come up with some of most atrocious untruths (still catchy, and pithy—vaccinations, anyone?) that thinking candidates can come up with. But there is some balance. Fact checkers abound, for instant on CNN and on many, many blogs on the internet. Almost every statement that a candidate makes, from the cost of Obama’s trip to India to the impact of vaccinations on child health is debated and fact-checked. And many a times, candidates are forced to admit they were wrong and retract their statements.

A system of checks and balances leads to some equipoise in the game of claims and counter-claims that is the battleground between the reductionists and the perfectionists. There is a bit of that on claims regarding low-income countries; some folks do make a valiant stab at debunking them (see for example the FAO work in box 2 here , Duncan Green’s blog entry on this and Sylvia Chant’s paper). But there are way too few. And every time the fact is repeated by yet another Very Knowledgeable Person, it is reified and gains an immortal life of its own, shorn of the moorings that tied it down to its original, possibly modest and appropriately caveated academic contribution (if there was one).

So, part of the role of institutions like The World Bank, should be to encourage groups that take on the duty of verifying the twitter-like feeds that come out of the speeches at the big global forum. Such a group would (a) distill and provide a set of twitter-facts that Very Knowledgeable Persons can use in their Very Important Speeches; (b) would act as a detector of completely incorrect facts so that myths do not perpetrate and would (c) be a living object that takes feedback, requests for fact-checks and comments on facts from users all over the world.

With the Open Data initiative at the Bank, it has become a lot easier for people to participate. What we now need, as President Zoellick of this institution suggested recently, is to democratize development. And what better way to start than by inviting development practitioners from all over the world to assist and guide global leaders on facts about their own countries and lives?

But where would such an organization be housed? We know that it is not going to come up organically, just by the virtue of the fact that it hasn’t. Simply put, the people who know typically don’t have the time, the inclination or the weight to influence the “fact-debate” on low-income countries. We also know that it is going to require researchers who are familiar with data from many countries around the world and are willing to provide what is, by its very definition, a public good (if the fact is bad, and the speech is public, it’s a public bad. Correcting it must be a public good). Options may include global think-tanks like the Centre for Global Development or Brookings Institutions, or the research department of the World Bank itself. 

Wherever it sits, it (a) has to be public and (b) has to respond to queries from the public comes in. Maybe the solution is some kind of development facts “wiki” -- where the crowd (sometimes) goes after what is grossly wrong.   But Wikipedia depends on dedicated folks who monitor it and shepherd it along.  

In the end, this democratized knowledge place may not be the best, and it may not always take us where either the reductionists or the perfectionists want us to go, but it may be better than what we have now.   And an informed debate of the issues – where careful empirical work and (fact based) grand statements mix -- can lead to durable policy, just look at conditional cash transfers versus what’s been done on women’s agricultural productivity. 

Do write in and let us know what you think:

  1. Do we have a problem with the “big” statements?
  2. What’s your favorite example of a development myth?
  3. Do you think that our system of checks and balances may work?
  4. Do you think that we should have a group that deals with these facts and puts out fact-checked twitter-length messages? Any nominations?

And oh, this blog just got 49,563 page-views, making it the most popular one on development ever.



Submitted by joe on
1. Yes - lies are lies. And if you tell lies to collect donations/support, you're not far away from producing a violent rape movie to promote your cause. And then when you've done that, where do you go next? 2. The whole 'minerals extraction is linked to rape in Congo' thing. 3. Nope. Don't forget we're talking about the public here - they have less attention-span than a goldfish. 4. Happy to try, if someone would pay me to do it. I doubt anyone would, hence not likely to happen.

Submitted by Anonymous on
By nature I am almost suspicious of any overly high concern for image. Image after all is something that one earns as a result of an effective product and delivering results and is lost for largely the same reasons. The Bank’s image can easily be attacked because much of the its work deals with ambiguity. While it is probably good to counter the most egregious and unfair attacks against the Bank, overall, I am not sure we should engage too much in peddling our ware. It is even worse when some of the slogans we propagate to make our case then are essentially incorrect or misrepresentations. Communication in slogans and sound bites is a practice common in politics, advertisement and unfortunately the tweeting world where complexity is abhorred in favor of pictures and a few words that cannot really capture the nuances that make up what really happens on the ground. While the modern media communication may help popularity and acceptance by the greater and a largely uninformed public, it does not give a true picture of what happens on the ground and of the challenges that remain unmet. In fact such communication can lead to misunderstanding that pose a real risk for the Bank to be accused of being untruthful. Development is not politics, nor advertisement, it affects the lives of thousands of people some of whom are winners and some are losers. While we have safeguards and take measures to limit the impact on losers, the fact is that there are some, and that cannot be changed, it is part of development. Consequently, for the Bank to tout its success in short sound bites is always fraught with risks as there is always another angle to the story. In the end is it more important to be popular in the public's eye or is it more important to be credible to our clients. I will always choose credibility over popularity, but then I am no politician and do not depend on my popularity for my survival.

Submitted by Anonymous on
I couldn't agree more with your comments. I hope that the vast majority of staff would agree especially with your last two sentences: "In the end is it more important to be popular in the public's eye or is it more important to be credible to our clients. I will always choose credibility over popularity, but then I am no politician and do not depend on my popularity for my survival."

Submitted by Malcolm on
I couldn't agree more with the sentiment of this blog: the number of well-intentioned gender "fictoids" floating around is flabbergasting, predictably resurfacing around any global meeting--including our own annual and spring meetings. The fact-checker idea is interesting. Perhaps it could be an added function (app?) of OECD's existing Wikigender tool/site? [] I could easily see it functioning with an advisory board sharing some of the burden, comprising someone from the OECD DCD (or DAC), the WB (the blog authors would appear to be prime candidates, along with colleagues from DECDG) and a handful others. Hopefully, a myth-busting list would grow organically after a while, reducing new time inputs, along with a wishlist of facts that *can* be found but haven't been compiled yet.

Submitted by Punam Chuhan-Pole on
Markus, Jishnu – Thanks for the blog post. I enjoyed reading it. I think you are not being innovative enough when it comes to checks and balances—i.e. fact checkers. Instead of relying on “development practitioners from all over the world to assist and guide global leaders on facts about their own countries and lives,” why not also tap into students (and universities) across the globe? With the Bank’s Open Data initiative, anyone with drive and some tech savvy can become a fact checker. Throw in a few incentives such as competitions to identify the biggest fact busters and you would be able to attract legions of youth to be part of this project. Once you have reached critical mass, the group should be sustainable. I suggest calling your group Fact Checkers Without Borders (hope the name is not taken). You spend some time agonizing over where the fact-checking organization should be housed. I would think a more important issue is one of sponsorship—i.e. which group(s) should be the (initial) sponsors of such an initiative. Equally important, I imagine, would be rules regarding content.

Submitted by Steve on
Too many words. Just #ThinkEqual.

Checking facts usually requires tracing sources, but that alone can be a challenge. My favorite example is the frequently heard claim that 70% of the world’s poor are women. It is hard to imagine how we might credibly estimate such a number given that household surveys typically do not tell us anything about distribution within the household, but that did not stop someone. The puzzle is where the number comes from. The earliest statement I can find is in the 1995 Human Development Report where it says that “Poverty has a woman's face-of 1.3 billion people in poverty, 70% are women” (p.4). But I can find no source in that report. Does anyone have any idea where this 70% figure comes from? Martin

Submitted by Kabs on
Some thoughts: Do we have a problem with the “big” statements? Yes What’s your favorite example of a development myth? That development research is going to play a big role in improving the lives of the poor. Do you think that our system of checks and balances may work? It certainly seems worth trying out. Punam's idea above (using students - cheapest skilled labour around) sounds interesting. Although the reason it works well in American politics is that the facts spewed by politicians are often checkable. This is not the case in development - if Dr. Ravaillon can't adequately trace this 70% stat, then what hope have mere students? Do you think that we should have a group that deals with these facts and puts out fact-checked twitter-length messages? Any nominations? Would be fun for development nerds. Not sure if it will have any significant effect on poverty alleviation. Would need strong institutional and financial backing to be even moderately effective.

Submitted by Anonymous on
It might be true that a statement made simply that glosses over all complexity can. make people get of the couch and move but a lot of harm has been done by the doers and shakers of the world. Consider how the Hindu right in India has gone out of the way to propagate the idea that Muslims are agree threat to India - stated with great passion, with no need felt to check facts, their statements can make Hindus feel vulnerable and ready to defend their faith - whereas a complex rendering of the fact that Muslims are a very differentiated community and that they have chosen to make India their home as well as the acknowledgment that every community has people who side with violence would make one pause to think. One could do "fact checking" and indeed students would be great resource but what is hard to counter are these highly affect laden statements that require constant battle against false propaganda. If one is attacher or a researcher, should one concentrate on generating research that will build a different picture of the situation or should one be doing fire-fighting operations all the time?

Submitted by Brett on
Related question: have you seen this post on the '4 million missing girls' number, and if so, what do you think?