There is a well-known idiom saying that you can't compare apples and oranges. But this is precisely the challenge researchers often face when it comes to measuring the jobs impact of development projects. Having standardized impact evaluation tools and methods is a milestone for private sector-led job investments, and it allows international financial institutions, development practitioners, and governments to build on existing knowledge to develop solutions. And this is precisely one of the goals that Let's Work partnership, composed of 30 different institutions, is currently pursuing; to track the number of jobs generated from private sector-led interventions, the quality of those jobs, and how inclusive those jobs are in a standardized way, so apples are compared to apples and oranges to oranges.
The need for a standard evaluation tool
It is common for researchers and development practitioners to face financial and time constraints while performing assessments or evaluating the impact of an intervention. It is also difficult to compare interventions across sectors, countries, and regions. These issues may be exacerbated when the intervention design is not fully informed, or the tools to implement the intervention are not harmonized. In these cases, evaluators often need to spend additional time and funds to understand the logic behind the evaluation while in the field. Additionally, comparing evaluations across organizations is usually difficult and many times not possible.
The handbook Let’s Work is developing will enable organizations and researchers to carry impact evaluations using standardized tools and methodologies, enabling comparisons of the short- to long-term impacts of private sector-led jobs investment across various sectors, countries and periods of time. By increasing the knowledge on what works, what doesn’t, and in what context, private sector-led job creation, job quality and job access efforts can be more effective and replicable.
An overview of our main findings so far:
The handbook is being developed based on the results of comprehensive value chains, tracer studies for measuring past interventions, and macro-simulation impact evaluation pilots conducted in eighteen countries over the past three years. We have implemented more than 30 case studies to estimate the job impact of investments focused on skills training, better working conditions, financial markets, and sectors such as agribusiness, construction, small and medium enterprises finance and infrastructure.
1. Standardizing the jobs impact measurement within Value Chains is Key. This method captures jobs effects across firms, meaning the direct and indirect jobs created within the value chain as well as jobs destroyed. We can quantify employment potential inclusive of the informal sector and identify possible constraints and solutions to lift those limitations.
We learned that from potatoes and waste recycling in Lebanon to aquaculture and poultry in Zambia, it is possible to have a standardized base guideline; however, the methodology still needs to be adjusted for specific economic, political and social contexts. This methodology is best used to measure jobs impacts in the agriculture sector, but not necessarily for the service or manufacturing sectors. The studies have helped us build a more rigorous set of guidelines, and the first part of these guidelines will be published in a few weeks.
2. The technical feasibility of conducting tracer studies is challenged by projects implemented too many years before evaluation. Tracer studies analyze in retrospective samples of former beneficiaries or participants of a project and what type of services worked better in the long run, to improve the design of future interventions. This method documents changes in employment and wages, and tracks the effectiveness of interventions in terms of high-quality jobs creation. While challenging, in the case of India, for example, we were able to track how SMEs invested the loans received. We learned that the main challenge of using tracers is not necessarily that the surveys would have had to use recall data, but that interventions were not designed with an evaluation framework in mind, making it impossible to produce reliable findings. This tool is more useful when used in a small geographic area with limited movement of the beneficiaries, especially when working with the informal sector.
3. Macro models are particularly relevant when making assessments of jobs impacts. This is the most comprehensive impact assessment tool for simulating indirect effects of supply chains and distribution networks and induced effects of a particular intervention or sector. Our studies cover a variety of methods, industry sectors, and country contexts and provide a characterization of jobs by gender and other variables. From power in Turkey to airports in the Caribbean, this model is mainly used in electricity, infrastructure and transportation projects and the case studies have given us a critical mass of knowledge in the power sector.
While collecting, and analyzing data to work towards a guideline for private-led jobs interventions’ impact evaluation, Let’s Work delivered more integrated knowledge to lift jobs constraints and tackle the challenge of creating more and better jobs. We learned from our partners how to have a greater impact, while also improving our understanding of key issues such as urbanization, fragile states, technology, gender, and migration. We have made much progress developing the technical aspects of impact measurement tools, and we continue working to provide a set of standard guidelines to support development practitioners, policymakers, and the private sector.
This post is part of a series of blogs covering the three methodologies being developed by Let’s Work to estimate direct, indirect, and induced effects of private sector interventions on jobs, accounting for both the quantity and quality of jobs. Related blogs in this series can be found here.
This work has been made possible through a grant from the World Bank’s Jobs Umbrella Trust Fund, which is supported by the Department for International Development/UK AID, the governments of Norway and Germany, the Austrian Development Agency, and Swedish Development Agency SIDA.
Current partners of Let’s Work include:
- African Development Bank Group (AFDB)
- Asian Development Bank Group (ADB)
- Austrian Federal Ministry of Finance (BMF)
- Department for International Development (DFID)
- European Investment Bank (EIB)
- 15 European development finance institutions (CDC, DEG, etc.)
- Inter-American Development Bank (IADB)
- International Finance Corporation (IFC)
- International Labor Organization (ILO)
- Islamic Corporation for the Development of the Private Sector (ICD)
- International Youth Foundation (IYF)
- Ministry of Foreign Affairs of Netherlands
- Overseas Development Institute (ODI)
- Private Infrastructure Development Group (PIDG)
- Swiss Secretariat for Economic Affairs (SECO)
- World Bank (WB)
- World Business Council for Sustainable Development (WBCSD)
Join the Conversation