World Bank Voices
Syndicate content

Pilots are for airplanes, not projects

Aleem Walji's picture
I recently heard a senior official from an African country emphatically say she wants nothing to do with pilot projects. There are too many of them she asserted and not enough to show for them. Pilots are for planes not projects, she said.

I can’t help but smile when I hear that because I mostly agree with her. The practice of global development looks like a cemetery of pilots that never grew, and sometimes pilots that died without even knowing what went wrong or why. That’s an entirely avoidable outcome, but one that requires a much more disciplined approach to data-driven experimentation and continuous, deliberative learning.

And yet I often hear my colleagues react negatively to the idea of experimentation when it comes to poverty alleviation. How can we ‘test’ solutions on poor communities who cannot afford the risk of failure? But that assumes that what we’re doing is working, that governments and non-profits are meeting the needs of the poor and that outcomes are largely positive. We know that’s not the case. So why are we shy about asserting that we need to try new things, test what is untested, and accept that many things will not succeed the first time but that’s not failure as long as we’re learning and improving what we do? 

If we want breakthrough innovation, we need to break a few things along the way and understand what went wrong. In the spirit of continuous learning, I think it’s important to distinguish between a pilot and a prototype. Piloting is when we start small, tinker, and later grapple with how we grow and scale impact. When things don’t go as planned, we look for people to blame and hide our failures. Scale is hardly ever reached because we don’t start with the end-game in mind.

Conversely, a prototype is when you thing big (from the start), start small, move fast, learn fast, and grow. It’s an experimental process driven by data, observation, continuous learning. It’s about a deliberate process that is iterative, modular, and designed to scale from the beginning.

Think of a ‘demonstration school’ in an impoverished African village where all the children are provided with textbooks, nutritious meals, and good quality teachers. When results are promising in the first village, the ‘pilot’ is lauded as a success but it’s entirely unclear how results will be replicated, whether the ‘solution’ is commensurate with the scale of the problem and how unit costs will be reduced over time.  Contrast this with a school-in-the box model developed in Kenya which trains hundreds of local instructors (most of whom are not formally trained teachers), leverages low-cost information technologies to deliver curricula and measure learning outcomes, and keeps costs to less than $8/student per month. The ‘prototype’ has a much higher chance of scaling across the country because it was designed from the beginning with the end in mind.  

It may sound obvious why the latter approach is better than the former. My experience, however, suggests that many development projects fall into this avoidable trap. If we’re going to solve social problems at the scale that’s required, we need to get this right. So let’s keep our pilots in airports and strive for something better in schools and hospitals.
 

Comments

Submitted by Anonymous on

Very nice.

Submitted by Alain Robinson on

Nice, I agree

Submitted by Eva Benita A. Tuzon on

In many development debates, there will always be that fallacy of arguments. The value of pilot projects are to test the strength of [one's] theory of change and be able to capture the knowledge generated from those interventions [impacts] provide counter factual, proceed replicating if evidence are right, terminate if it seems unworthy for further waste of resource. Pilot projects are in fact a thread of research where any sector cannot afford to play on people's lives [trial and error], hence the necessity of becoming "precisely correct" than "absolutely wrong."

Submitted by dan mwanje on

You are spot on Eva. I think the writer should instead have endeavored to help the (African leader)to admit failure on her part to manage pilot projects successfully. I suppose that leader could have failed to deliver and, instead of admitting failure and seek jelp on how to improve, she chose to hide behind rhetoric (pilots are for aeroplanes).

Submitted by Boukari Tare, on

A pilot project is always restricted and not taking into account the context of the project site. Inexperience professional s use it to find their way out in the development instituons. The numerous pilot projects implemented in the developing countries can cover the needs of Africa or Latin America in teems of resources. It is a waste of money. It is delaying the scaling up of most Developing countries. Why not learn from previous pilot projects and go for prototype? I am sure it can be done particularly in social services. It is obvious that NGO and most of bilateral aids and risk management gurus would not step in. They prefer micro instead of macro which could take developing countries to be in queue for the next two hundreds years. Hopefully decision makers both in the developing countries and the World Bank will come together to a win win approach while designing projects.

Submitted by Aleem on

Thanks for your comments. I make a clear distinction between a 'pilot' and a 'prototype'. I certainly agree that it's important to test hypotheses before rolling things out in a big way. But it's important to design programs keeping in mind the scale of the end goal. Too often, we don't pay enough attention to the 'end state' when we design pilots. When we do start with the end in mind and iterate along the way, we design very differently (that's my assertion).

My goal is to minimize the graveyard of pilots while still taking small risks along the way. Try fast fail fast is another way to put this. I think it's possible to think big and start small. But to do this well, we must design for scale from the start.

Submitted by Fred Perry Fort Lauderdale on

I agree with you Aleem .. trying and failing is not bad but making the trials in wrong way is bad. So to be successful even trials must be in the right direction.

Add new comment

Plain text

  • Allowed HTML tags: <br> <p>
  • Lines and paragraphs break automatically.