I. The problem of comparing apples and oranges
Comparison of countries lies at the heart of assessing financial sector performance. In doing so, analysts often simply compare financial sector indicators such as credit to the private sector as a percentage of GDP for a given country to a regional average or a set of "representative" countries.
However, such comparisons are only accurate to the extent that the selected benchmark is appropriate. In practice, countries often differ substantially in terms of structural factors that affect financial development. Thus, a simple comparison can lead to inaccurate conclusions.
Figure 1 below displays a simplified example that demonstrates the core of the issue. It shows dots that represent countries with different “structural factors” (e.g. population density) plotted against their “financial development”, i.e. the extent to which the financial sector fosters economic growth via better risk sharing and more productive investments. The figure shows that in terms of financial development, Country B is better than Country A in an absolute sense.
Yet, simply concluding that County B performs better than Country A obscures relevant policy insights. The reason is that once “structural factors” are accounted for and countries are compared to their appropriate, structure-specific benchmarks (the blue line), Country A is a relative star, while Country B is a relative underperformer.
Figure 1 – Accounting for structure is key to benchmarking countries’ financial sectors
II. Which structural factors should one account for?
Econometric research, based on historical, world-wide data, indicates that the following factors matter for indicators that capture the development of the banking sector, debt and equity markets, and non-bank financial institutions: economic development and population and demographical characteristics (population size, population density, age distribution metrics, etc.). Other factors such as whether the country is an offshore financial center or has an oil-based economy also matter.
Note that we do not account for factors that directly capture financial policy. Instead, the objective is to account for factors that are outside the government’s direct control (at least in the short run). The reason is that once structural factors are “filtered” out of the financial indicator, the remainder is a reflection of the quality of policy (i.e. the distance to the blue line in figure 1).
There is one complicating factor, however. Although we consider economic development to be a structural factor, financial sector development in part also follows economic growth. This can affect the benchmark. Yet, if one plausibly assumes that financial policy affects economic development with a time lag, the difference between the observed value and the benchmark should still contain information about the quality of policy.
Figure 2 – Statistical benchmarking can also reveal performance misclassification
III. Statistical benchmarking in action: A country example
As mentioned previously, comparing a country’s financial indicators to ad hoc benchmarks such (as medians or averages of) economic or regional peers can be misleading. To illustrate this, Figure 2 compares Costa Rica’s private credit to GDP ratio to the Latin America regional median and its statistical benchmark, which accounts for structural factors (the expected median in the graph).
By regional standards, Costa Rica seems to be on track or even performs slightly better than the region since 2004 (ignoring credit quality considerations). However, according to the statistical benchmark measure, Costa Rica’s private credit to GDP ratio has been well below the benchmark in the last 10 years, implying that it systematically performs below its reasonable potential.
In conclusion, the example shows that statistical benchmarking is better capable of dealing with the heterogeneity between countries and can thus reveal policy insights that would remain hidden with conventional benchmark approaches -- even when one compares to countries in the same region or income group.
Join the Conversation