“The most calamitous failures of prediction usually have a lot in common. We focus on those signals that tell a story about the world as we would like it to be, not how it really is. We ignore the risks that are hardest to measure, even when they pose the greatest threats...We abhor uncertainty even when it is an irreducible part of the problem we are trying to solve.” (Silver 2012)
Each year governments invest billions of dollars towards long-term development. Yet their investment decisions are engulfed in deep uncertainty – about long-term demography, economic growth, technological developments, cost of energy, the impact of climate change, and a host of other factors. Deep uncertainties are difficult to acknowledge, understand, and manage. We are more comfortable facing risks we can quantify and solving problems for which we have familiar, well-honed tools. Compounding the problem, analysts and decision makers routinely face pressure to demonstrate that a decision is risk-free. Political and cultural expediency presses them to ignore rather than acknowledge uncertainty and present their decision as advantageous and certain. Such thinking can keep us in the dark about the real threats to our decision, and may lead us to make brittle decisions that fail when the future surprises us.
These challenges compel us to revisit our beliefs about what makes a decision “good.” Instead of ignoring uncertainty, we should try to understand the threats it may pose to our choices and make decisions that are robust to an unpredictable future. Through the practical demonstration in our paper, "Making informed investment decisions in an uncertain world," we seek to motivate and equip analysts to do just this. In particular, we answer three questions:
1. How are deep uncertainties currently managed in lending projects?
2. Can new approaches help projects better manage those uncertainties?
3. Are these approaches practical, and what challenges do they pose?
First we review the economic analysis of ten World Bank projects approved between 2002 and 2011 in order to understand how they managed risk and uncertainty. Our review suggests that projects recognize and seek to manage uncertainty, but may lack the right methods to do so. Instead, they typically use traditional approaches in which investment decisions are optimized to a projected set of future conditions. In doing so, they may not assess whether the selected investment is sound if the future unfolds differently.
Next, we test and demonstrate the practicality and value-added of Robust Decision Making (RDM) to the economic analysis of a prior World Bank project. In 2006, the Electricity Generation Rehabilitation and Restructuring Project sought to improve Turkey’s energy security in part by increasing near-term energy supply. We use the same data and models utilized in the original analysis, but in a different way. Rather than seeking to inform electricity investments in Turkey with predictions of the future, we seek to inform them with assessments of their robustness to an unpredictable future.
The original decision was to rehabilitate an existing coal plant. Other options included building new coal-fired, gas-fired, or other power plants. Decision makers had two key goals: a) to produce electricity at lowest cost possible, and b) to ensure a rate of return of at least 12%*. We evaluated each option according to these same metrics in hundreds of plausible future states of the world that varied under different sources of deep uncertainty that would affect project performance. These include future wholesale price of electricity, capital cost, length of construction time, and discount rate. We used the results to answer a series of specific and useful questions:
• How do decision makers’ options perform across a wide range of potential future conditions?
• Under what specific conditions does the leading option fail to meet decision makers’ goals?
• Are those conditions sufficiently likely that decision makers should choose a different option that is more robust?
Our RDM analysis determined that rehabilitating the existing local-lignite plant would meet decision makers’ objectives in most futures, but not all. Critically, it revealed that, out of all deep uncertainties, a specific combination of input energy costs would make it more desirable to construct a new gas-fired plant. However, these energy price conditions seem less plausible than the conditions that would threaten the success of a new gas-fired plant. This knowledge could reasonably lead decision makers to conclude that rehabilitating the existing plant is the most robust option available.
Methods like RDM can provide decision makers with much more salient information about the merits and vulnerabilities of different options than the traditional combination of cost-benefit and sensitivity analysis. This can focus decision makers’ attention on the uncertainties that matter most to a decision, and avoid gridlock over less important ones. It can make them aware of key tradeoffs and of actions they could take to reduce their vulnerability. Ultimately, it puts the decision back in the hands of decision makers by helping them take measured risks and be less vulnerable to surprise.
The demonstration further suggests that we can readily incorporate robustness analyses into cost-benefit and other economic analyses that analysts perform every day, by using our same data and models differently. While our analysis focuses on a World Bank case study, we believe methods like RDM hold value for governments and the broader development community.
Changing how we make development decisions requires a cultural shift as much as it requires an analytical shift. It is a long road. Yet methodological innovations like RDM can help. By motivating and equipping analysts to manage uncertainty, they can shape how we think about, discuss, and ultimately make decisions.
*At the time of this project, change mitigation was not a widespread concern, and this analysis reflects the original priorities of decision makers related to energy security. The same analysis today would almost certainly include climate change impacts and other concerns.
Join the Conversation