What are we learning about the impacts of public works programs on employment and violence? Early findings from ongoing evaluations in fragile states

Labor-intensive public works (LIPW) programs are a popular policy intended to provide temporary employment opportunities to vulnerable populations through work-intensive projects, such as the development and maintenance of local infrastructure, that do not require special skills. For a review of LIPW programs (design, evidence and implementation), see Subbarao et al. here. In fragile states, LIPW programs are also presumed to contribute to social and political stability. The developed infrastructure allows for the implementation of other development and peacekeeping activities, while employment opportunities may help prevent at-risk youth from being recruited by armed groups. Despite their popularity and presumed impact on beneficiaries, the evidence base of LIPW programs has been surprisingly weak.
 
The Development Impact Evaluation (DIME) unit, in collaboration with the Fragility, Conflict and Violence Cross Cutting Solutions Area (FCV-CSSA) and the Social Protection and Labor Global Practice (SPL-GP), is carrying out a multi-country set of 7 Randomized Control Trials (RCTs) of LIPW programs targeting around 40,000 households across 5 countries: Comoros, the Democratic Republic of Congo, Côte d’Ivoire, Egypt, and Tunisia. This initiative is part of a broader research program on Fragility, Conflict and Violence (FCV) — a portfolio of 35 impact evaluations in over 25 countries that focuses on 5 key priority areas: (i) jobs for the poor and at-risk youth; (ii) public sector governance/civil service reforms; (ii) political economy of post-conflict reconstruction; (iv) gender-based violence; and (v) urban crime and violence.

These LIPW programs are as diverse as the countries in which they are implemented. (See details on each country’s LIPW program and related IE design here.) However, they share two key features that are central to the evaluation initiative: (i) the creation or upgrading/repair of community assets (e.g., social and/or economic infrastructure); and (ii) the provision of a wage to beneficiaries in exchange of work on any such community infrastructure, albeit temporarily. Our IE studies—carried out since 2014—sought to ascertain the impacts of LIPW programs on economic welfare in the short-term and in the long-run as well as on social stability broadly defined to encompass anti-social behaviour (e.g., lawlessness and violence) and pro-social attitudes and behaviours (e.g., civic engagement). The results are already available for 4 of the 7 IEs.
 
So, what do we find? In a nutshell, the results are strikingly consistent across IE cases. In the short-term, the LIPW programs all deliver critical economic benefits for the communities where they are offered, whether measured by consumption, income or rates of employment. However, we find little evidence (so far) of an impact of LIPW programs on economic welfare outcomes in the long-run or on a wide range of outcome related to violence and social stability at the micro level. Nevertheless, there is still potential to redesign these social safety net programs through better targeting of beneficiaries, and add-on interventions such as capital infusion, training and mentoring, and behavioural therapy.
 
Result #1: LIPW programs have positive impacts on economic welfare in the short-term
 Focusing on employment, in Côte d’Ivoire, total employment increases by 12 percentage points (pp) as a result of the program. This is driven by impacts on wage employment, which increased by 44 pp and self-employment which decreased by 9 pp. In Egypt, the two LIPW projects—the community infrastructure and the community social services show differing results. In the former, where the baseline employment levels are as high as 89%, we find that the program had no impact on employment. However, in the case of the community social services, respondents who were assigned treatment were significantly, i.e. 15 pp, more likely to have current employment. In Tunisia, total employment is 8 pp higher as a result of the program, mostly driven by increases in the informal sector. Moreover, the effect on employment increases to 11pp when the individual surveyed was the head of the household.
From the evidence, it seems that these workers do actually get jobs but do they earn more? There is some hope. Across Africa and the Middle East, incomes show increase— about 42% in Côte d’Ivoire, 35% in the Egypt Services and 26% in Tunisia. The LIWPs not only boost incomes but this is also translated, in several cases, into higher expenditure and savings. For example, in Tunisia, spending on rent over past month is 112% higher and expenditures on home improvement over past year is 52% higher for program beneficiaries. In Egypt, non-food spending is 11% higher for program beneficiaries. In Côte d’Ivoire, amount of savings in terms of four-month stock is $65 more for program beneficiaries whereas in Egypt, the propensity to save is just over double in the past 30 days for program beneficiaries.  

Figure 1. Short-term impacts of LIPW programs


Result #2: Economic welfare impacts of LIPW programs in the medium/long-run seem limited
 Unfortunately, thus far, the results on long-run impacts across the different cases are available only in the case of Côte d’Ivoire and seem less promising. Specifically, the study measured impacts 12-15 months after program completion and finds no evidence of impact on employment levels or composition and some impact on earnings. However, one finding shines bright like a diamond- there are some effects on heterogeneity. The most vulnerable who gained the most in the short term also benefit the most in the medium term. Furthermore, we do have some indicative results from one of the two Egypt studies, the community social services, on the extent to which short-term impacts on welfare outcomes last overtime. We exploit variation in the length of time between project completion and when the survey was taken and find an inverse relationship (See right hand-side panel in figure 2 below). That is, employment effects tended to subside the more time elapsed since project completion, suggesting limited persistent effects overtime.
 
Figure 2. Long-run/persistent impacts of LIPW programs



Result #3: There is little evidence of an impact of LIPW programs on local crime and violence
What do we measure? It’s important to clarify this because the civil conflict literature typically focuses on “armed rebellions” or “inter-personal/ community conflicts” usually motivated by ‘social causes’, whereas the literature on crime and violence tends to focus on “anti-social behaviour” or “lawlessness” usually defined in reference to a particular legal framework. In a post-revolution context like Egypt or Tunisia, however, both may be important. Thus, we looked at an array of phenomena, including petty crimes such as theft as well as more serious acts of violence such as physical aggression or armed robberies, among other outcomes. We also looked at various measures of pro-social behavioural or attitudes such as inter-personal trust, civic engagement, perceptions of government legitimacy, and pro-democratic norms (e.g., support for no-violence in politics).
What do we find? LIPW programs investigated have no consistent impact on short term violence or civic engagement i.e. 1-6 months after project completion. Across our range of outcomes including physical violence, emotional violence, theft and armed robbery, we find no consistent differences between treatment and control communities. Only in case of the Egypt infrastructure project, we find a 5 pp decrease in physical violence, but not across other evaluations. Similarly, we find no significant differences in civic engagement indicators including trust in government and willingness to participate in democratic processes.

Figure 3. Impacts of LIPW programs on social stability

What do we make of these results?
On one hand, from an optimistic angle, consistent findings of positive impacts of LIPW programs on measures of economic welfare even if only in the short-term is encouraging and suggests that they fulfil at least their social protection function, as safety nets. However, if the goal is to generate long-term sustained impacts, it may be necessary to complement LIPW programs with add-on interventions such as capital infusions and/or cash grants and test whether these help sustain benefits.

On the other hand, the absence of evidence of an impact on measures related to crime, violence and civic engagement is worrisome. One might be tempted to attribute the null findings to the peculiar nature of the interventions under evaluations (i.e., the fact that these are short-term emergency programs) and/or the context in which the supply or demand of violence is relatively low (i.e., the country investigated are in the Middle-Eastern and North African region and the levels of violence at baseline are relatively low.) However, we try to rule out this hypothesis in two ways. First, we conducted disaggregated analysis for “at-risk youth” (i.e., under 30 years) versus other beneficiary groups on the theory that the former is more likely to engage in anti-social behaviour. We find no statistically meaningful differences. Second, we look at the evidence for other types of employment interventions (e.g., entrepreneurship programs) that targeted ‘at risk youth’ in settings typically considered to be more prone to armed insurgencies and violence (e.g., Burundi, Liberia and Uganda, among other countries- Gilligan et al. 2012; Blattman & Annan 2016; Blattman et al. 2013) and that tend to have much longer time-horizons. The evidence is surprisingly consistent with the evidence from our own studies: these programs had positive impacts on economic welfare outcomes, but still had no effect on reducing crime and violence or on pro-democratic norms and attitudes.

It is also possible that these results are simply reflecting the relatively weak first stage and do not necessarily constitute evidence against the opportunity cost theory. The idea behind the opportunity cost theory is that employment opportunities and earnings in “peaceful” activities would provide strong disincentive to engage in lawlessness and ‘anti-social’ behaviours. But if earnings from “peaceful” activities are not high enough then it is entirely possible some people might continue to engage in criminal or armed activities to supplement their income.

Last, but not least, another possibility could be the inadequacy of the opportunity cost theory, which typically assumes economic or material incentives as primary motivators for anti-social or violent behaviours. Yet we know from social science theories that other factors, including behavioural constraints, socio-political grievances or perception of injustice may also be at play (Gilligan 2016). If true, then no amount of employment and earnings can effectively address the issue of crime, conflict and violence in fragile states. One implication of this insight is that to be more effective, development interventions need to consider a wide-range of drivers of conflict and violence, rather than just economic/material incentives, and address these in a targeted way.
 
Concluding thoughts
While the findings presented here are preliminary, and we need to wait for more data to be collected to measure outcomes over longer-term horizons, some lessons are emerging for both policy and future research.

1. Labor intensive public works programs can at least provide a short-term social protection function. Such benefits are not trivial, especially given the lacking or inadequate social protection systems in many fragile states. Moreover, aside from the immediate short-term economic benefits, LIPW programs may provide fragile state governments much needed space to implement more pro-growth labour market policies. However, the seemingly lack of sustained impacts in the long-run suggests the limits of LIPW programs as instruments to lift people out of poverty and vulnerability.  
2. Improving the targeting and implementation modalities of LIPW programs is a key area for future work. The need for better targeting is well illustrated in the case of the Côte d’Ivoire LIPW program. Cost-benefit simulations found that when the LIPW is designed to target the most vulnerable and reduce costs outside transfers (such as administrative costs), it would become much more cost-effective, i.e. break even in about 3 years versus 23 years for running the intervention under the status quo. Relatedly, the importance of implementation modalities is illustrated by the contrasting evaluation results from the two components of ELIIP case in Egypt. The community infrastructure component was carried out by private contractors/firms that not only hired workers who seemed to have an employment but also some form of income generating activities already, and as a result, it is likely the program only helped to displace or crowd out other economic opportunities. By contrast, the community social services component was implemented by NGOs and appears to have done a better job targeting workers who were unemployed at the time and it seems there was a genuine expansion of labor supply, especially for the young girls who otherwise would not have worked. This suggests that the intensive versus extensive margin seems important for LIPW programs. 
3. Other policies may offer more potential for long-term poverty reduction and for reducing conflict and violence. Public works programs may be a useful tool for governments to quickly provide employment opportunities to many individuals for a short-term, but for longer-term progression out of poverty, alternative solutions such as graduation programs may have more impact (see Markus’s blog here). Likewise, there are other promising interventions to reduce violence that include psychological and social support. Behavioral interventions such as those evaluated in Chicago (Becoming a Man program, One Summer Chicago Plus program) and Liberia seem promising, though questions remain about their scalability across a diverse set of situations. 
4. There is also the need for better measurement of conflict and violence, particularly in primary data collection. For one thing, government restrictions might prevent the collection of information on important indicators. (e.g. in Egypt, the Central Agency for Public Mobilization and Statistics must review and provide permission for all survey questions relating to security and public demonstrations and the tendency has been to remove all questions related to sensitive issues such as conflict, violence, political demonstrations, etc.) Even in the absence of any such restrictions, the sensitivity of this subject could lead to reporting biases from respondents. Some techniques such as list or endorsement experiments, as well as the diversification of sources of data on conflict and violence (e.g., administrative, third-party or satellite data) could help mitigate the issue of missing data associated with the volatility and unpredictability of FCV settings. 
5. Finally, more research is also needed on the macro-micro challenge/connection in this area of study. Most empirical studies assess intermediate effects at the micro-level—E.g., are individuals employed? Yet participation in armed groups and violence may be driven by macro-level logics—E.g., do rebellions form? Understanding these linkages could help target appropriate interventions at the ‘right’ locus of control or level of decision-making.

The findings and policy implications of these multi-country LIPW interventions have been presented at a Policy Research Talk at the Bank targeted towards the operations teams and senior directors. See here for the Policy Research Talk video and PowerPoint and here for the newsletter highlighting the key points of the talk. I gratefully acknowledge contributions made by members of the DIME FCV team including: Brooke Alexis Braswell, Julie Claire Moranne Bousquet, Rohit Chhabra, Sarah Elise Elven, Chloe Fernandez, Nausheen Khan, Besa Rizvanolli and Tomomi Tanaka.