What gets measured gets done: Evidence from court

Are judicial reforms worth doing? It turns out, we cannot be sure, but we have a story to tell about a reform, its impact, and the impact of having measured that impact.

The literature. The evidence suggests that faster justice is important for development. Studies show that slow contract enforcement proceedings reduces gains from trade (North 1991, Djankov et al. 2008).  Measures of the investment climate such as those found in the Doing Business Report reward legal reforms aimed to simplify procedures and speed up adjudication. In short, there is ample evidence showing that building up a solid justice system is core development work.

Okay, so speed is good, but how do we actually do that – speed up court decisions? There is not much formal court-level evidence out there, that legal reforms could get the job done (La Porta et al. 2008). Judges are powerful, independent, and perform many different tasks. Thus, it is not clear that changing the rules of the game would affect their speed. Some recent evidence shows that imposing performance contracts on these types of bureaucrats can in fact be harmful (Finan et al. 2017, Rasul and Rogger 2017). Another issue in measuring the impact of reform on judges’ behaviour is that measuring judges’ performance is not so straightforward. In sum, there was not much to go on by ways of evidence on the impact of procedural reforms. This was until some recent work we started under the auspices of the Economic Governance Project in Senegal.

The Senegal context. At the onset of the project, Senegal’s contract enforcement period as measured in Doing Business was slow. Senegal was ranked 146 on the ease of enforcing contracts in the Doing Business Report (2014). The Economic Governance Project aimed to speed up civil and commercial adjudication to improve the investment climate.

The impact evaluation. The impact evaluation started at a workshop to identify priorities for evaluation among Senegalese government programs. First order questions from the government team were, what are the main bottlenecks in the civil and commercial procedure? The bulk of the case-level data was all in paper format (here and here), with the digital court database recording only date of entry. A cross-disciplinary team expended large amounts of time and effort to ensure that these data retraced the full history of all civil and commercial cases, down to each hearing.

These data helped the team from the Ministry shape their reform agenda: “The evaluation work was useful throughout the reform process, especially in drafting the decree n° 2013-1071 (6 August 2013) which modified the decree n° 64-572 (30 July 1964) of the civil procedure code. The baseline data helped us identify bottlenecks along the procedure and motivated a reform of the preliminary hearings.” (Bienvenu Moussa Habib Dione, Director, Direction des Affaires Civiles et du Sceau).

Can changing the rules of the game affect judges’ performance? The reform (Decree n^o2013-1071) gave judges the duty and powers to conclude pre-trial proceedings in four months. At baseline, pre-trial hearings lasted on average 157 days, or over two-thirds of the total length of court proceedings (Kondylis and Stein 2018). The decree allowed judges to desk-reject cases based on insufficient evidence in the first hearing.

Kondylis and Stein (2018) evaluate the impact of this decree on the efficiency and quality of civil and commercial adjudication in the Regional First Instance Court of Dakar. The application of the decree was staggered across the six chambers of the Court over 2013 and 2014. The impact evaluation combined that with high-frequency caseload data to capture the causal impact of the decree.

Results. Did this simple procedural reform change the behaviour of skilled, independent, and powerful judges? Cases that entered the court after the introduction of the decree completed their pre-trial procedure about 43 days faster than those that entered before (Figure 1). These cases are 23 percentage points more likely to end pre-trial stages in under four months.



What do judges do differently to achieve these speed gains? They use their new powers to desk-reject cases that present extremely weak evidence at the first hearing (Figure 2).



They are also more likely to fast-track well-documented cases straight to deliberations. This is a straight cut in procedural formalism: desk rejections and fast-tracking account for 44% of the reduction in delays, and cases experienced on average two fewer pre-trial hearings from a baseline i.e. before the reform of 8.3. However, desk rejections and fast-tracking are not the entire story. Judges were more likely to apply pressure on parties by issuing strict deadlines for adjournments (Figure 3).



Using the procedure, appeal data, and text from the judicial decisions, Kondylis and Stein (2018) do not measure any significant change in markers of quality of the procedure. Interviewing firms that were involved in disputes over our study period and applying a stated preference framework suggest that firms see a positive value in faster proceedings. While more work is needed to assess the overall welfare and distributional implications this reform, this work shows that giving judges higher discretionary powers leads them to change their behaviours, and that firms seem to benefit from these changes.

The case for evaluating reforms: An epilogue. The epilogue we are about to tell demonstrates the importance of rigorously measuring the impact of reforms. After the final results of the study were shared, the high-level magistrates in attendance could not believe that the reform had such impact. In fact these results were used further: “The evaluation results helped us elaborate a new law n° 2017-27 (28 June 2017) on the creation, organization and functioning of the new commercial tribunals and commercial appeal chambers. These results contributed to improve Senegal’s Doing Business ranking and created a momentum for this reform process” (Bienvenu Moussa Habib Dione).

To us, the main message is this. What gets measured gets done. Quantifiable measurement of the impact of this reform helped not only communicate progress, improve the country’s ranking, and shape the way the new Commercial Tribunal is organized. In conclusion, reforms may be good, but evaluating them is even better!