The disciplining effect of supervisory scrutiny in the EU-wide stress test

Since the financial crisis, stress tests have become an important supervisory and financial stability tool. Against this background, a question is whether stress tests contribute to financial stability by promoting risk reduction in the banking sector as recent evidence suggests. Stress tests offer deep insights into banks’ vulnerabilities to supervisors and the public through an intense supervisory process. In a recent paper, we show that higher supervisory scrutiny led to a disciplining effect for banks that were part of the 2016 EU-wide stress test, coordinated by the European Banking Authority (EBA) and conducted by the European Central Bank (ECB).

How supervisory scrutiny is exerted in stress tests

In Europe, stress tests involve interactions between banks and supervisors on banks' risk management practices as well as confidential communications about best stress-testing practices and techniques. We use data on these confidential interactions to approximate how much scrutiny was exerted on each bank under the direct supervision of Single Supervisory Mechanism (SSM) during the 2016 EU-wide stress test. These interactions arise as part of the constrained bottom-up approach pursued in the EBA-coordinated exercises (see figure 1). In this context, banks use their own internal models to generate projections, for example, for credit losses. Meanwhile, banks' projections are challenged by the competent supervisory authorities typically by applying top-down models and other challenger tools. In the presence of material deviations between these two sets of projections, “flags” are triggered and later discussed between supervisors and banks. Banks need to comply with or explain the issues raised in the interactions with the ECB. We construct a scrutiny measure by counting the flags related to credit risk projections. Intuitively, banks that received more flags had to work harder on their resubmissions and had lengthier and probably more intense interactions with supervisors, while banks that received no flags in principle had no further interactions with supervisors.

Figure 1: Simplified illustration of one quality assurance cycle under the constrained bottom-up approach.

Source: Own illustration based on Mirza, H. and Zochowski, D. (Macroprudential Bulletin Issue 3, Ch.2, 2017).

Scrutiny measures the intensity of stress tests

We apply a differences-in-differences approach where we use the stress test as a treatment and the involved scrutiny as a measure of the intensity of the treatment. In the first step, we compare the credit risks of banks that were part of the stress test and banks that were not part of the stress test four quarters before and four quarters after the 2016 stress test. In the second step, we compare the credit risk of banks that were subject to a more intense supervisory scrutiny and banks that received less or none.¹ The 2016 EU-wide stress test was executed on significant institutions (SIs). Less significant institutions (LSIs) were not tested and we therefore use them as the control group.²

Effect of supervisory scrutiny on credit risk

We focus our analysis on credit risk that accounts for a large part of the stress testing projections and on average for 86 percent of risk exposure amounts in bank balance sheets. To measure credit risk at the bank level, we use the risk-weight density (RWD), that is, the aggregate risk weight assigned to total credit risk exposures according to regulatory standards.

Figure 2: Estimate and 90 percent confidence interval of the differential effect on RWD between tested and non-tested banks before and after the reported quarter.

First, we find no significant difference in RWD between the treatment and control groups before the stress test (2015q1 and 2015q4, see figure 2) but significant negative differences for the period after the test (2017q1 to 2017q4). The reduction in RWD of tested banks after the stress test was on average 4.2 percentage points lower than the reduction of not-tested banks. This effect is economically material as it amounts to a change of about 20 percent of the standard deviation of RWD.³ These results confirm the findings based on US data that “treating” banks with stress tests can affect their risk. 

Figure 3: Marginal effect and average effect estimates with 90 percent confidence intervals of the differential effect of scrutiny intensity on RWD.

Second, we show that the more interactions banks had with supervisors, the higher was their reduction in RWD after the stress test exercise (see figure 3). We find that those banks that received more scrutiny (the half with scrutiny intensity above the median) exhibit 5.6 percentage points higher decrease in credit risk than the half that received less scrutiny.⁴ All in all, these findings provide novel evidence that the tighter and more intrusive supervisory scrutiny associated with the EU-wide stress-test has the potential to enhance banks' risk management practices and induce lower bank risk. 

Policy implications

We contribute to the emerging evidence on the effectiveness of supervisory scrutiny. Our results suggest that stress tests that are conducted by applying a robust quality assurance of banks' projections and models have disciplining effects on stress tested banks’ risk. However, one of the stress tests' primary objectives is to assess banks' risk profiles correctly. Our findings do not provide information on how well this objective is met. The possible strategic underreporting of banks' vulnerabilities under a bottom-up approach could undermine the reliability of the stress test outcomes from this perspective. Pursuing a more unbiased top-down approach while retaining supervisory interactions with banks during and after the stress test might be more suitable to achieve this goal. Therefore, with our analysis, we only deliver one insight among many that could serve the policy discussion on the design of future stress tests in Europe.