Syndicate content

Surprising Results from Fragile States

Joel Hellman's picture

If there is one thing that most of the donor community can believe in, it is this:  aid gets better results in countries with better governance.  The data linking aid effectiveness and governance go back to the work of David Dollar and Craig Burnside around 15 years ago.  And though there have been many challenges to the original findings, more recent studies by Aart Kray and colleagues on the World Bank’s (WB) own portfolio confirm that over the past 25 years, WB projects have performed better in countries with better governance.  But for most people, it’s not the data that convinces them of this simple, but powerful maxim; it’s just a matter of plain common sense.  Or is it?

Something strange and unexpected has happened to the WB’s portfolio in the last few years.  Since 2009, projects in fragile and conflict affected states (FCS) have out-performed projects in the rest of the portfolio as judged by both internal and independent evaluations. The share of “satisfactory or better” projects has been 5-10 percentage points higher in FCS versus non-FCS over a three year moving average.  Of course, a few years of volatile project performance data are not enough to challenge one of our most deeply held assumptions about aid performance.  But what is going on here?

Spoiler alert:  I don’t have the answer, just a lot of potential hypotheses.  Here are some that come to my mind.  You’ll undoubtedly have others.


A young girl in Gaza FCS projects are getting more staff time and resources.  That’s true.  Over the past few years, the rate of increase of WB staff on the ground in FCS and budgets for FCS projects is much higher than in non-FCS. We are now spending three times more of the WB’s own budget in FCS versus non-FCS for every dollar of IDA assistance provided.  This translates into much higher growth in budgets for project design and supervision. But this also raises difficult issues of potential trade-offs, since in the flat budget environment of recent years, we are taking money out of the non-FCS portfolio to cover the higher implementation costs in FCS, potentially “robbing Peter to pay Paul.”

FCS projects are getting simpler in design and less ambitious in results.  For years, we have been telling teams to stop over-designing projects.  Keep it simple.  Define results that can be achieved during the project’s lifespan.  These lessons may have sunk in better in FCS, where they are so patently obvious. In the better governed countries, our ambitions may still get the better of us.  Certainly, projects in FCS are smaller.  We don’t have a simple measure of their design complexity or ambitiousness of their results.  We should. This might also explain why better performing FCS projects with their less ambitious outcomes are not necessarily leading to better performing FCS countries in terms of poverty reduction.

Projects in FCS are implemented differently than in non-FCS.  “Using country systems” is all the rage in the aid community.  Clearly, it makes sense that if we want to build up government capacity for sustainable results, we need to route aid projects through government systems and stop using special project implementation teams (PIUs) and special fiduciary rules.  Of course, WB projects go through government systems in all countries, including FCS.  But we are clearly going the extra mile in those places with better governed country systems.  Perhaps the more traditional approach to project implementation is getter better immediate results in FCS than in those countries with greater reliance on country systems. If this is true, it has important implications for the country systems debate.

Projects are evaluated differently in FCS than non-FCS. A more mundane, but equally plausible, hypothesis is that we are either less rigorous or perhaps more generous in evaluating projects in FCS.  With weaker statistical systems in FCS, we have less data to monitor results.  This could either lead to expected outcomes that are more difficult to falsify.  Or given the higher risks in FCS, it might lead to greater willingness among project staff and evaluators to “look the other way” in FCS projects and to accept higher variation between the results expected at the start of the project and those ultimately achieved in light of the risks.

We need to get the bottom of this, since there are important implications of each hypothesis. And we have no idea if these recent trends will be maintained over time, so we need to understand what is driving them. There is also the question that though we may increasingly be doing projects right in FCS, we might not be doing the right projects to address the fundamental drivers of conflict and fragility, and hence poverty reduction, in FCS.

But regardless of what is driving these results, they raise questions about some of our most deeply rooted assumptions about aid effectiveness.  Many important aid funds are allocated according to formulas heavily weighted on governance indicators.  Many still wonder whether we can get results in FCS.  Many still say that increasing aid to FCS is “throwing good money after bad.” Three years of volatile project performance don’t constitute a trend and we’ll need to know much more before we draw any conclusions for practice.  But for now, let’s at least change the way we talk about these FCS countries.  Let’s not assume that there is an inherent trade-off between poverty and performance in aid allocation.  Let’s not assume that more aid to FCS will necessarily lead to poorer results. Let’s recognize that if we want to end extreme poverty, we need to be ahead of the curve to address the increasing concentration of the extreme poor in FCS.  By testing the hypotheses above, we might even be able to draw lessons from FCS to improve the performance of all aid projects. Now that would be a switch.
 

Comments

Submitted by Geir Sundet on

It can also be that FCS projects are "first generation" projects that simply seek to put systems in place. An evaluation can then find that the system is in place, ergo success.

In non-FCS settings more projects are second or third or more generations, and it has become obvious that the systems aren't working. Will be interesting to see the next wave of FCS evaluations to see what has happened with the current successes.

Submitted by Joel S Hellman on

You are right that the content of FCS projects might be different and that might explain the gap between FCS and non FCS projects. But that still wouldn't explain the improvements in the FCS portfolio over time. Perhaps we have gotten better at designing first generation projects. At the same time, we might be struggling more with the second and third generation projects.

Submitted by Geir Sundet on

Makes sense. Good to see all the hard thinking about programming in FCSs paying off. The next bit will be harder. Any signs that the Bank is listening to Andrews, Pritchett and others on their recommendations on Problem Driven Iterative Approach and a more problem oriented and flexible approach to programming?

Submitted by Joel S Hellman on

We are indeed listening to Matt, Lant and Michael on PDIA and hinking very hard about how to operationalize their approach. Not so easy in the organizational context of the Bank, as they themselves recognize.

Submitted by Geir Sundet on

Indeed. Just saw this from Matt on why the Bank is finding it not so easy. http://matthewandrews.typepad.com/the_limits_of_institution/2013/10/can-the-world-bank-do-pdia.html

Submitted by Sheelagh Stewart on

Extremely interesting. I prefer the hypothesis about the design of appropriately ambitious projects in the WB

Submitted by Dawit on

Very interesting. Could it also be that the projects are having the side effect of building state capacity. This finding might indicate a need for reexamining the whole concept of FCS.

This is a brilliant post on a critical question.

A few thoughts:

- The improved performance of projects in fragile states may not be so new. Page 116 of the IEG report, “Engaging with Fragile States” shows that as early as 2005, the share of projects in fragile states rated satisfactory on outcome exceeded that in stable countries. (Report available here: http://lnweb90.worldbank.org/oed/oeddoclib.nsf/24cc3bb1f94ae11c85256808006a0046/a4d6461b0067e049852571f500551e1b/$FILE/licus.pdf)

- It is interesting to note that under the Bank’s early country allocation models, portfolio performance was given considerably more weighting than it is now: around 33% compared to 8% today. Fragile states (and the Bank’s country office staff) would therefore have been “rewarded” for the performance improvement you identify.

- I suspect your first hypothesis – staff time and resources – can explain some of the story. This would be consistent with evidence of various correlates of project ratings that have been identified: frequency and staffing of supervision missions; quality of economic and social analysis preceding the project; quality of project monitoring and evaluation.

- One hypothesis missing from your list is that the normal timeframe within which projects are evaluated may be inappropriate for obtaining a true assessment of impact in fragile settings. Fragile states are, by definition, prone to bouts of instability, and it seems implausible that the legacy or sustainability of successful projects would be immune to significant deteriorations in the environment such as a violent conflict or a government being overthrown. There may therefore be a case for reassessing project impact 5 or 10 years after completion.

Finally, for anyone interested in reading more on this topic, you may enjoy this brief I wrote in 2011: http://www.brookings.edu/~/media/research/files/papers/2011/11/fragile%20states%20chandy/11_fragile_states_chandy.pdf

Laurence

Submitted by Joel S Hellman on

Laurence,

Thanks for the great comments. Two quick responses:

-On portfolio performance as an indicator in aid allocation formulas: I have been arguing for some time that this indicator should be given far more weight. It seems to me that we have been using governance (as measured by the CPIA rating) as a proxy for the performance of aid projects (under the assumption that weak governance will undermine the quality of aid projects) and yet we do have a direct measure of aid performance in the portfolio quality indicator. If we are interested ultimately in results, which we believe will come from better performing aid projects, then we should link allocations to demonstrated performance. Some might argue that aid allocations should reward better governed countries because good governance is a "good" in and of itself regardless of its impact on aid performance. That's fine, but we would need to be explicit about what we are trying to achieve through our aid allocation formulas -- getting maximum results or incentivizing countries to improve their governance.

-On the missing hypothesis, you are absolutely right. It may very well be that even though we "do projects right" (in terms of getting results), we are not doing the "right projects" that address the fundamental drivers of conflict and fragility. This could put the results of even successful projects at risk -- perhaps not right away but over time. This is a very important hypothesis that we need to investigate seriously.

And I recommend to others that they read your excellent brief from 2011.

Joel

Submitted by Miria Pigato on

Surprising Results? Think again!

My unit works on 13 West-African countries. Most of them are classified as fragile or have experienced conflict and insecurity in recent years (e.g. Cote d’Ivoire, Togo, Chad, Mali, Benin, Guinea Bissau, Guinea, Mauritania, and Niger). Senegal, Burkina Faso, and The Gambia are relatively stable. Cape Verde is a middle-income country. We provide technical assistance, economic and sector work and Development Policy Operations (DPOs) supporting Poverty Reduction Strategies.

Are we helping our clients to deliver results? We decided to compare the findings from the 2012 Bank wide DPO review, which covers most budget support operations in the Bank since 2009, with those from our own countries. Guess what: 66% of results were achieved in West Africa as compared to 60% for the Bank as a whole.

For us, this finding was somewhat unexpected. Performance in West Africa, particularly in poverty reduction and governance has generally not been very good. Thus, we would have expected our DPOs to under-perform the Bank more broadly.

In order to get some perspective on this apparent contradiction, we looked in detail at the performance of Benin, a country that has experienced political instability and social unrest but has a CPIA marginally above the cut-off for fragile countries. During the last 7-8 years, Benin benefitted from a total of 6 budget support operations (PRSCs, all of them putting great emphasis on public financial management and procurement reforms.

How did Benin perform? Extremely well, better than West Africa and the rest of the Bank! About 86% of the results were achieved. And yet, several governance indicators in Benin have not improved. The 2012 Worldwide Governance Indicators report shows that since the 1990s progress in Benin has been marginal, or indeed non-existent.

Just to summarize: despite our own great success in achieving results, there has been no meaningful change in Benin’s overall governance performance. Therefore there must be somewhere a disconnect between our supported actions, the stated results, and real change in society. By the way, this is not an uncommon situation. Read, for example, the recently released PEFA for Niger. It says, upfront, that since 2008 “the measurement of the public financial management performance indicators shows that progress has been insignificant. Of the 31 indicators, 21 were ranked the same or lower than their 2008 level”. Certainly not for lack of assistance from donors, the IMF and us.

Why are we not making real and meaningful change?

First, an analysis of the ‘prior actions’ for Benin shows that they might not have been robust or ambitious enough to make a meaningful difference either to governance or to service delivery For example, undertaking budget audits that have been outstanding for 7 or 8 years, limiting the number of payment orders, aligning the budget with the MTEF, or passing this decree or that decree, are all important in their own right. But does the approval of a decree creating a regulatory agency actually lead to improved regulatory oversight and enforcement? One could look at many of these prior actions and question whether they were sufficient to introduce real and sustainable change in governance. Second, many of the prior actions did not necessarily form a coherent reform package. For example, PRSC-4 included a prior action related to submission of a new procurement code. The new code was approved in 2009, but the subsequent PRSC’s 5 and 6 did not follow up with actions to ensure that the code was actually implemented and enforced. Third, there is limited capacity in Benin across, including in the central ministries involved with public financial management issues. Most of our prior actions focus on getting the policy framework right, but this is insufficient in environments of weak capacity – additional complementary support in the form of technical assistance is often also required in order to fully achieve desired results.

Finally, the choice of results may not have been overly ambitious or necessarily well-targeted. Reducing payment and procurement delays may suggest some improvement in administrative efficiency. The more meaningful outcomes, those related to improved governance, reduced corruption, lower procurement costs, were not stated as expected results. Had they been included as results, they may not have been met.

Needless to say, we are now adopting a different approach, one that addresses the many issues described above. Will let you know how it goes…..

Hope this helps

Miria

Submitted by Joel S Hellman on

Miria,

The gap between project level results and country level outcomes is a critical issue in FCS. You could interpret the gap in 2 ways (at least). On the one hand, since real institutional change has a much, much longer time frame than any project lifespan (ESP in FCS), it is no surprise that project level results only make marginal progress in changing slow moving institutional change indicators like PEFA. All we should expect from projects is to set realistic goals on the path towards institutional change and keep chipping away at the problem. On the other hand, you could argue that excessive focus on immediate results has led to project designs that are insufficiently ambitious (ESP in FCS) and look to make simple changes that can be achieved but have little impact. The latter argument suggests a need for us to think much more seriously about the results agenda in FCS. Is our desire to demonstrate results in the short term channeling our efforts into less transformative engagements where we are unwilling to take real risks? This is an extremely important issue in FCS that needs to be at the center of our debates. So your Benin case really does push us to "think again" as you say.

Submitted by Stella Ayika on

I have been keenly reading the comments here and wondering if Nigeria should not be classified as one of West Africa's fragile states with the violence in recent years?

Submitted by Shanta on

Joel, I agree with Laurence and others that this is a great post. I do have one issue, which may strengthen your results. The definition of a "satisfactory" project is one that met its objectives. It doesn't say anything about the counterfactual--what would the place have looked like in the absence of the project? The distinction may be particularly relevant in fragile states, where the situation in the country may have been deteriorating, but the project prevented an even greater deterioration. So the fact that project objectives in fragile states may have become less ambitious is not such a bad thing if we compare these objectives to the counterfactual. By contrast, in stable countries, a project may have achieved its objectives, but the country may have achieved those results without the project too

Submitted by Nadia Molenaers on

Interesting debate !! Just a question: if projects are not necessarily strengthening country-systems (which is a bit suggested), then what exactly is being achieved that is sustainable in the long-run? It's not just about addressing the root causes of conflict, also about the institutional environment being able to promote economic growth and to deliver services and security to citizens. As long as parallel systems are being used, I don't see how these interventions are improving governance systems in place, which ultimately are the main drivers of development. And as such higher satisfactory results (in the short run) may very well be about achieving the less ambitious goals, but not walking very far in terms of institutional progress.
Maybe public opinion, or the fashionable ideas pushing 'value for money' might also push aid agencies to go for the low-hanging fruit, rather then venturing into much needed but riskier interventions in the institutional arena.

Submitted by Joel S Hellman on

Nadia,

There is an unresolved tension in the donor community on the country systems issue. More use of country systems will inevitably impact project level results. And more use of country systems exposes donors to greater country risks. So we can't have it all. There are trade offs and some of the recent data on project performance in FCS and non FCS highlight those potential trade offs. But perhaps once we can measure those trade offs in terms of the impact on project level results, we can compare those impacts to the long term benefits of institutional strenghtening. It's time for us to be much more explicit about the trade offs.

Its not just the World Bank. The findings echo our (Agulhas Applied Knowledge's) analysis of the performance of DFID's portfolio undertaken for them in 2007-9. This was only for internal consumption at the time and its unlikely it was widely shared. Our personal views were that such better performance scores may have resulted from greater care taken in design, scrutiny and closer management of programming, coupled to the modality akin to the issues you mention; interventions were more project based, tended to be more clearly time bound and often had more specifically identified outputs. I'm not sure we saw any positive bias from scorers; indeed, in comparison with non FS countries, there tended to be a more stakeholders involved in assessment, which can often results in better quality assessments of performance.

Submitted by Joel S Hellman on

Fascinating to hear that DFID found similar results. I would be interested to hear from other donors as well.

Dear Joel, very interesting hypotheses indeed.

While I do not work on FCS I am wondering to what extend there is also an alternative hypothesis. Crises normally bring change. Or at least this is what one seldom hears in policy discussion in settings where there is systemic corruption. When confronted with the question of what would bring a change in the situation, answers often are that a crisis will bring change.

Would this sense of "better aid results" be also due to the urgency to get results and that the "bar" on FCS is much lower than on non-FCS? That it, the elasticity of the measure varies. That is, ceteris paribus, a big result in a FCS will be equal to a small result in non-FCS?

Submitted by Joel S Hellman on

Jairo, It is certainly plausible that FCS projects could be achieving better results because in a post crisis situation, smaller interventions could have bigger results given how far things might have fallen as a result of the crisis. Just as growth rates in post-crisis situations can be greater than more stable countries due to the rebound effect, the same could be said for other results in FCS projects. One wonders if we could test for such an effect.

Submitted by Vanessa Wyeth on

Really interesting discussion. I'm especially curious to hear about efforts to explore the points that Laurence and Miria raise -- e.g. is the whole (in terms of improved longer-term governance outcomes) greater than the sum of its parts (success at the project level)? Of not, what would that tell us about whether we are doing the right things? And are we making adequate efforts to ask and answer this question?

One possible implication for the current discussion going in the Bank about scorecards, concerns the definition of "success". Thinks to Joel's important point about whether the Bank is "doing the right projects well", rather than just "doing projects well". Our argument at International Alert would be that there's a need to redefine success in FCS, to integrate progress in reducing fragility and violent conflict - even in seemingly technical efforts like infrastructure investments. Perhaps, in line with Joel's arguments hypotheses, this trend tells us that the Bank is automatically and organically shifting the goalposts to define success in easier-to-achieve ways. But that doesn't necessarily equate to picking the right outcomes and goals. Reducing fragility is a different enterprise, and one fraught with challenges for the Bank, but it is the right thing to do in FCS as Joel says, and scorecards used to define success and failure need to incorporate it.

Submitted by Joel S Hellman on

Phil, excellent point. We were hoping to try to use the New Deal state and peacebuilding indicators as an input into our scorecard, but the g7+ have not yet endorsed a common set of indicators. It would be great to have indicators owned by the FCS themselves as these are political indicators that must gave full country ownership.

Submitted by Chris Pycroft on

Joel, interesting post and great debate. It am particularly interested in your sentence: "There is also the question that though we may increasingly be doing projects right in FCS, we might not be doing the right projects to address the fundamental drivers of conflict and fragility, and hence poverty reduction, in FCS." This, for me, is the critical aspect. There may be an assumption that doing a few, relatively "simple" things in FCSs - health provision, for example - will yield good returns in terms of results. We may be picking this up now in terms of project performance. But once these "low hanging fruit" have been plucked - what then? And a focus on results that does not address the route causes of instability, fragility and conflict may actually do harm in terms of sustainability and accountability. You are right that we need to be cautious. What looks like success may not be. What looks like failure may not be either.

Submitted by Joel S Hellman on

Chris, good to hear from you! I couldn't agree more that this is the critical issue. As happy as I am to see the data on improved project performance, I am still very concerned about what ths means for overall outcomes in FCS. I do worry that the "mania" for tangible results in the short term could be deflecting us from a tougher institutional development agenda -- especially in critical areas for fragility like justice and accountability -- that do not lend themselves to bit-by-bit log frames. We are looking at the content of the World Bank portfolio to see if we can detect any changes in the types of activities we have undertaken over time to test whether we are moving towards "projects of least resistance" (a less sanguine term for low hanging fruit). Though we know that short term measures to build confidence are critical to securing coalitions that are the foundation for peace, we also know that the sustainability of these efforts is linked to the progress of the institutional development agenda. This needs to lead us into long term relationships with clients to build a foundation for institution-building. But such relationships are fraught with risk and only show the most incremental of results. Though most everyone agrees with this analysis, they still push for an approach to results that shifts us away from this institutional agenda. Perhaps if we can document how the content of the projects is changing over time, we could galvanize action. Right now, so many of these issues are based on conjecture and anecdote. Let's use a detailed analysis of the project portfolio over time to try to see if we can document trends empirically, calculate the pros and cons,and make some informed decisions about risks and results in our FCS work.

Really interesting blog - some great testable hypotheses here.
While some may argue about how new these results are I think there are many more people that are unaware and it is great that you’ve highlighted these results. And interesting to see that Nigel Thornton came to the same conclusion with DFID projects. I also like your hypotheses not least because they are testable and I look forward to hearing the results of your further studies.
I’m inclined to agree with Laurence Chandy that we need a long term perspective before we can be sure these results are wiped out by a return to conflict.
There are two other elated research questions. First has the international community has got better at reducing the risk of the return to violence – ie a change of external factor is the reason for the improvement in FCAS scores. And second is there a difference between the various types of FCS e.g. some states are FCs due to chronic low CPIA scores others to a past conflict. It would be interesting to see what is the result of looking just at g7+ states for example.
As to the question of use of country systems of course the whole premise of the New Deal is that this is a critical part of statebuilding. There would be an extraordinary policy choice to be made if the research concluded that a more traditional approach led to higher project success but slowed the rate of statebuilding. As you say we need to worry about “doing the right projects” as well as “doing projects right”. More prosaically getting the definitions right for the research on use of country systems will be difficult. The Afghanistan Reconstruction Trust Fund in effect used country systems as it reimbursed government spending. But most technical measure won’t pick up this critical nuance.
Finally as you mention interest and challenge around taking forward PDIA approach you might like to know that ODI has been running an innovative TA programme using a PDIA approach which is showing promising early results. The independent mid-term evaluation (by Agulhas) rated the project in the top 25% of all DFID funded projects, and noted that the project was “an embodiment of the approach to Technical Assistance known as ‘problem-driven iterative adaptation” and that the approach adopted “appears to have a higher success rate than is typical for PFM capacity-building programmes”. All papers are available on www.budgetstrengthening.org. ODI has been discussing these results with Matt Andrews given the read across to his research and he is now joining the Advisory Board on this project.
Look forward to staying in touch on all this as Alastair McKechnie and I will be taking up these issues in a chapter we’re contributing to a new book from Brookings (editors Homi Kharas and Laurence Chandy)

Submitted by Joel S Hellman on

Marcus,

Thanks for the comments. You make an important point about the risk of violence and how it might be an important determinant of more successful projects. That raises the question of whether the shifts in the risk of violence are independent of the development investments or are caused by them. Both are plausible and we need to try to unpack the causation. It is interesting that a lot of work of the randomistas seems to show that even the positive outcomes of a lot of CDD engagements (in terms of employment and community infrastructure) did not have much of an impact on social cohesion or levels of trust/cooperation at the village level. This work is not specifically targetted at FCS engagements (though there has been work on the National Solidarity Program in Afghanistan), but it does suggest that maybe external factors are much more important as a determinant of the risks of violence as opposed ti the direct impact of developmenrt interventions.
On differences between types of FCS, we are looking carefully at these issues in the next round of research differentiating between "always fragile" countries and that those that move in and out of the FCS classification. The World Bank Board has now requested us to review the entire approach for identifying and classifying FCS. This will be a good opportunity to identify different types of fragility.
Finally on country systems: I would not want this discussion to lead us backwards on country systems. But I do think that a better, empirically driven identification of the trade offs will help us to make define risks more realistically and to make more informed choices about project implementation.
We have a lot of work to do to explore these different hypotheses. And I hope we can do some of this work across the portfolios of a number of key donors with FCS projects. It would be great to get ODI involved.

Submitted by Claire Leigh on

Interesting piece. A couple of comments from me:
-The fact that the revelation about better results in FCS is based on a moving average of all FCS projects may disguise something important. Projects in FCS can fail spectacularly in some cases, and succeed spectacularly in others. Ie. ‘high risk/high return’ trade-offs are in play. This may mean that risk-averse development partners are no more inclined to intervene in FCS, despite higher average ‘success’ rates.
-The most interesting line in the whole piece is this: ‘Perhaps the more traditional approach to project implementation is getter better immediate results in FCS than in those countries with greater reliance on country systems. If this is true, it has important implications for the country systems debate.’ This seems like a pretty radical departure from the received wisdom, certainly from within fragile states themselves. But you also say that CS are used elsewhere in order to ‘build up government capacity for sustainable results’. In other words you acknowledge the long-term benefits of using CS. But the longer term benefits seem to have been factored out in this hypothesis. ie If it’s really true that use of CS improves capacity and sustainability, this should be included as a mark of success in projects where CS are used, and as a sign of failure in projects where CS are not used. If that particular hypothesis is plausible under current WB success measures, it implies the success criteria are wrong.

Submitted by Joel S Hellman on

Claire,
Thanks for the interesting comments. On your first point, you are absolutely right that project performance ratings in FCS have longer "tails" i.e., there are more instances of very low scores and very high scores than in other countries. This is not surpising giving the higher risks. And this could very much affect risk averse donors. If we had a better method for calibrating our projects by the risk undertaken, then there would be much more willingness of the donors to take those risk if their returns are evaluated in light of the risks undertaken.

On your second point, I wanted to make very clear in the original blog that there is indeed a trade-off related to country systems. Though there may be an initial impact on project performance, it might be outweighed by the sustainability gains from strengthening country systems. We just need to be explicit about the trade offs. I would not advocate turning back from country systems. Indeed, we have much, much further to go in this area. But I do think we should move further with a clear understanding of how it might impact project performance in the short term and what we could to mitigate those effects.

Submitted by Khetsiwe Dlamini on

This is a good thought provoking discussion. We definitely want to be ahead of the curve in FCS if we are serious about ending extreme poverty. We know that there is an increasing concentration of the extreme poor in FCS.

I agree with the approach suggested by Joel of avoiding the “ugly duckling” label for FCS work. Aid effectiveness in more difficult environments is the way to approach the evaluation of performance in FCS. Aid effectiveness is a well-researched area which FCS workers can take more advantage of and couching FCS work in along the continuum of aid effectiveness helps normalize this type of work. It might help establish FCS work as one of the categories of the science of delivery that we need to continue to sharpen global knowledge and practice in on rather than the “odd ball” that is we feel good about being seen doing something/ any old thing in. I also agree that is need to factor drivers of peace and stability (the opposite of drivers of conflict) in FCS. Peace and stability is after all a prerequisite to project success – is it not?

A few reactions to the hypotheses offered in the initial blog.

Simpler Projects? I would like to think that designing simpler projects is just one part of the equation. I think we need to go further to consider designing projects more incrementally, whilst maintaining relevance and accessibility of the project components. Starting small and building up from there is an old small business tactic which can easily be applied to FCS projects. This approach helps manage risks, allows the project to build on its achievements and enables the transfer of knowledge to subsequent components. It should be fine to have projects that may seem complex in retrospect as a result of having built additional components in based on success. Too often complexity is front loaded and projects are designed in ways that stretch way beyond client capacity, at times even beyond the Bank’s capacity to deliver; and too many valiant attempts to squeeze complexity into inadequate time frames and budgets are still rewarded with project approval.

Projects are implemented and evaluated differently in FCS. “As they should be” is my immediate response. My experience of operations in the FCS is however that we put in too much effort into implementing and evaluating FCS projects in the same way that we do in non-FCS as an organization. The flexibility in process and procedure introduced by OPCS in response to client and country team complaints is not always taken advantage of. We tend to get stuck in the “normal” way/ rut of doing business giving to little consideration to the drivers of conflict as killer risks to the work we do. Lightening the administrative burden of FCS work would release more disposable time for client facing work which can reap the biggest rewards in terms of client relationship management and building country capacity. Increased capacity potentially creates more business for the Bank Group as it feeds client demand for more solutions. I think that we shoot ourselves in the foot if we are not trying to implement and evaluate FCS projects differently.

Submitted by Joel S Hellman on

I agree with both of your reactions. We certainly need to think about spreading risk in FCS engagements by undertaking more innovative projects that are scalable if successful and easy to drop mid-course if not. Sometimes I worry that the emphasis on transformational projects will lead us in the opposite direction. But small, scalable projects can become transformational.

And as for your points on evaluating FCS projects differently, we really need to have a risk adjusted model for evaluating results. I don't understand why we bother rating risks if we don't use those ratings to measure the impact of our results. If not, we will always go for "low hanging fruits" and avoid the really bold moves that entail high risks.

Submitted by Culture Clash on

Could it be as simple as ... life in war torn countries sucks so anything is better than what they have, ergo any project in FCS becomes transformational (or whatever the latest jargon du jour). To put it more bluntly, if you're starving, day old bread tastes great. If you eat regularly, day old bread tastes stale. There is so much academic double talk in these comments to make the whole discussion absurd, and call into question the purpose of the Bank at all.

Submitted by Koffi Alinon on

I am reading with interest the ideas on this blog. Congratulations Mr Hellman.
Beyond the economic explanations you can find about best project performance observed in FCS, I am adding some plausible facts from Eastern DRCongo:
- investment in "soft" interventions like capacity building in alternative disputes resolution and inter-community dialogue are likely to pave the way for "hard" stabilization projects
- there are funny examples of markets/administrative offices built but where no one want (or threatened) to use
- the lesson learned is to propose integrated community development approaches which can strengthen population resilience. The system experienced is to systematically address rising land conflicts and provide security of access to land and NR and come alongside with agricultural support as well as RGA for diversity of livelihoods. It is an integrated land approach implemented by UN-Habitat/FAO/UNDP.
One prerequisite that is also stressed by your note is that government bodies (here local governments at district level) should progressively own the process for its sustainability.
Would appreciate to share more document with you on this.

I'm new to the blog world but I'm trying to get started and create
my own. Do you require any coding expertise to make your own blog?
Any help would be greatly appreciated!

Add new comment