This blog is part two of a two-part blog series. The previous blog describes the Brazilian context and the difficulties with purely outcome-based key performance indicators (KPIs). This blog suggests an alternative approach, a model-based KPI, and illustrates what it could mean for the Legal Amazon.
Model-based KPI = observed outcome – expected outcome
In an upcoming publication “An Economic Memorandum for the Brazilian Amazon” we introduce the concept of a model-based KPI. The key idea here is that outcome and performance are not the same. Performance is the difference between an observed outcome and would have been expected, based on a forward-looking statistical model.
How such a model would look like, what variables it contains, will depend on the use case. If we go back to the five-year SLB example during the pandemic, the model should account for factors that are relevant to the outcome of interest but are not under the control of the issuer. These are exogenous factors. In the case of transport greenhouse gas (GHG) emissions, the global demand for air transport could be an exogenous factor. Since it fell drastically during the pandemic, expected GHG emissions would also have fallen. Factors also need to be exogenous because the issuer should not be able to manipulate them.
In order to qualify for a KPI, a model needs to be transparent, robust, and evidence-based, and must produce two types of outputs: first, an expected outcome which serves as the benchmark, and second, an outcome distribution that describes how likely alternative outcomes would have been. Figure 2 visualizes how these model outputs can be used to construct a KPI.
Figure 2: Anatomy of a model-based KPI
Having an underlying model not only makes performance calculation a simple task: knowing how likely alternative outcomes are allows us to assess ex ante, how ambitious an SPT is, and ex post, how strong a KPI is. To elaborate – if the actual observed outcome were just slightly better than the expected outcome in Figure 2, the KPI would be very low and only serve as a weak signal for actual performance. If the outcome were instead at the green bar, then the KPI would be very high and send a strong signal that the issuer indeed performed much better than expected. The strength of KPIs can then be tied to financial incentives in the form of step-ups or step-downs.
Table 1: KPI strength and financial incentives
The cutoffs (50%, 95%) can vary depending on the usecase.
Confidence intervals |
Within 50% |
Between 50% and 95% |
Outside of 95% |
KPI strength |
Weak |
Normal |
Strong |
Step-ups or step-downs |
+/- 0 bps |
+/- 12.5 bps |
+/- 25 bps |
Can models help us tease out performance? Back to the Amazon
In the same publication, we also explored what a model-based KPI could look for the Amazon. Previous research suggests that deforestation is due to two factors: economics and institutions. When demand for agricultural or forestry products picks up, the rate of illegal logging follows suit. At the same time, Brazil has put in place impressive instruments to protect forests, even though enforcement of existing laws has waned recently. Both economic factors and political will are needed to bring down deforestation.
Figure 3 shows how lagged global commodity prices, their volatilities, and the real effective exchange rate can predict deforestation. These factors were selected based on a macroeconomic model and machine learning methods. It is important to stress that the factors are exogenous, as previously discussed. Since the Amazonian economy is only about 10% of Brazilian GDP, it is unlikely to drive global market prices. However, global demand determines how profitable it is to clear forests for cattle or soybeans.
The results are striking and consistent with research. Deforestation was significantly lower than what the economic factors would have suggested after Brazil introduced ambitious forest protection measures (green flags in 2004, 2005, 2009 and 2012). In these years, an SLB would have rewarded the Brazilian government. Conversely, when the narrative around protecting the Amazon changed around 2018/19, no more incentives would have been provided—in fact, a step-up would have been justified in 2019. Our model also suggests that deforestation would have accelerated between 2018 and 2021 for economic reasons. Yet, it accelerated even faster in reality, which could be attributed to government action.
Figure 3: Better or worse than expected?
At the end of the day, issuers want to be sure that their efforts are adequately rewarded, and investors want to know that they are having a real impact. Otherwise, SLBs will face the same questions about greenwashing and additionality. A KPI based on a transparent, scientific model addresses these concerns by construction. Certainly, a model-based KPI is more complex than an absolute KPI. But having a model also ensures that issuers articulate clearly what in their view counts as performance – and what doesn’t. Without this separation, it would be difficult for investors to assess ambition and monitor actual impact. Issuers also benefit as a clearly defined KPI gives credibility to their selection of KPIs and sends a strong signal about their commitment to sustainability.
Join the Conversation