Five of the eight AGI pilots were able to successfully embed a rigorous impact evaluation design. We also had a centralized research team that ensured standardization of the research objectives and methods as much as possible. You can access the papers from the individual pilots on our website, and you can download useful documents such as our evaluation concept notes, list of core indicators, and survey instrument in our Resource Guide.
Here are some key recommendations for further research:
Unbundle evaluation designs and provide cost-benefit information by project component. AGI evaluations weren’t able to compare the relative impact of technical training versus life skills training or measure the impacts of specific project strategies, such as mentoring or placement assistance. Similarly, we can say very little about disaggregated costs of these components.
Have a cash-only evaluation arm. Youth employment interventions of all kinds are under pressure to demonstrate that their impacts are larger than what could be achieved through giving cash directly. See, for example, this relevant blog post from Chris Blattman. As part of this agenda we also need to understand the differential impacts of cash provision on young men and women.
Determine the optimal composition, intensity, and delivery of different mixes of skills. This is particularly true for life skills training, which tends to be much more heterogeneous across contexts and is far less expensive to implement than technical or business skills. Related questions around the appropriate age to focus on different types of skills and whether training works better in sex-segregated classrooms will aid in designing the next generation of youth employment programs.
Test strategies for job placement. Progress has been made in improving the delivery of skills training and in helping youth start businesses, but much less is known about how to cost-effectively assist youth to find and retain wage jobs. Interventions—some implemented in AGI pilots—that deserve more testing include:
- Variations in the length and intensity of job placement support: Most AGI interventions included three to five months of placement support;
- Performance-based contracts for the training providers, as used in both the Liberia and Nepal AGI pilots, though these have not been tested rigorously;
- Wage subsidies, as tested among young female community college graduates in the Jordan AGI, which achieved significant short-term gains but no long-term impact;
- Partnerships with large firms to create custom training programs.
Untangle the relationships between young women’s labor and health outcomes. The AGIs in Liberia and Nepal, using a technical and vocational education and training (TVET) model, did not have significant impacts on sexual behaviors or health outcomes, while the Uganda girls' club-based approach dramatically lowered fertility and increased condom use. One distinguishing factor about the Uganda project was that it worked with younger girls, starting at age 14. Another important question to answer is whether there is an optimal age threshold or whether there are other conditions under which skills training projects can affect sexual behaviors.
Bettina Jenny and Sonja Hofstetter of Helvetas Swiss Intercooperation, Switzerland and Gisela Keller of Helvetas USA explain how a skills development program in Nepal has trained over 100,000 youth— with more than 75,000 of them gainfully employed.
In Nepal, about 500,000 young people enter the Nepalese labor market every year. Most of them are unskilled and have not completed formal education. Moreover, the private sector in Nepal is weak, and a ten-year-long civil war (1996-2006) and subsequent ongoing political instability have contributed to the worsening economic and social situation. In short, getting a job is a huge challenge for many young people in Nepal.
Good intentions are not enough. Future employment and earning outcomes are the key indicator to measure the success of skills training. Many development actors provide skills training with the goal of making personal and economic perspectives available to youth in countries with high unemployment. Such programs tend to focus more on training delivery than on employment, and graduates of these kinds of youth skills programs often discover that their newly acquired skills do not meet market demands.
We do things differently. In 2007, the Swiss Agency for Development and Cooperation (SDC) joined with HELVETAS Swiss Intercooperation to establish the Employment Fund to create new and effective ways to scale up approaches addressing the alarming scope of youth unemployment in Nepal. Funded by the Swiss Agency for Development and Cooperation, the Department for International Development (DFID) and the World Bank, the Employment Fund began its operations in 2008. The Employment Fund offers training in about 80 occupations in construction, hospitality, garments and textile, agriculture, and electronics – to name a few -- in locations all over Nepal. The Employment Fund received a prize for good practice in youth employment awarded by the International Labour Organisation (ILO), and was among the ten finalists for the OECD DAC Prize for Taking Development Innovation to Scale.
Unlike many other skills training programs, the Employment Fund applies a results-based financing approach that has proven to effectively lead to gainful employment upon the completion of training. Training providers are paid based on their success in training youth and subsequently connecting them with the labor market. The key result is gainful employment.
As we today mark UN Women’s Day, it is worth considering what the inequality between men and women costs South Asian countries and what can be done about it.
One big cost of inequality is that South Asian economies do not reach their full potential. In Bangladesh, for example, women account for most unpaid work, and are overrepresented in the low productivity informal sector and among the poor. Raising the female employment rate could contribute significantly to Bangladesh achieving its goal in 2021 of becoming a middle-income country. Yet even middle-income countries in South Asia could prosper from more women in the workforce. Women represent only 34 percent of the employed population in Sri Lanka, a figure that has remained static for decades.
Improvements in the education and health of women have been linked to better outcomes for their children in countries as varied as Nepal and Pakistan. In India, giving power to women at the local government level led to increases in public services, such as water and sanitation.
Just as the costs of inequality are huge, so is the challenge in overcoming it. The gaps in opportunity between men and women are the product of pervasive and stubborn social norms that privilege men’s and boys’ access to opportunities and resources over women’s and girls’.
The Adolescent Girls Initiative (AGI) through its eight pilots taught us a great deal about how to make skills training more female-friendly and support young women's transition to productive employment. In addition to all the lessons we learned about working with young women, the pilots also taught us a lot about how to improve the overall quality of skills training.
Our Top 5 quality enhancement lessons, along with links to more information, can be found online in our Resource Guide and here:
Lesson 1: Skills training projects need to set realistic expectations for self-employment versus wage employment. In contexts with limited opportunities for wage employment, skills training projects should help orient youth to the likelihood of self-employment and develop content suitable to different levels of aspiration in that sphere. In Liberia, for example, we offered a job skills track and a business skills track. We ended up having to gradually reduce the size of the job skills track from 35 percent of trainees in Round 1 to just 18 percent in Round 3 after our impact evaluation showed the employment rate in the business skills track was much stronger. This wasn’t easy—it involved changing the orientation of the client, the training providers, and the girls themselves.
Lesson 2: Involving the private sector can improve the market relevance and overall effectiveness of training. AGI pilots partnered with the private sector in the implementation arrangements by hiring private companies to provide training tailored to the needs of a specific firm/sector—as in the Rwanda AGI—and by hiring private sector training and employment service companies to deliver training and assist with job placement—as in Haiti, Liberia, and Nepal. We also took low-cost steps to engage the private sector throughout implementation. For example, the Liberia AGI organized Private Sector Working Groups to provide routine guidance on project activities and enlisted members of the private sector to inspire the trainees by serving as guest speakers in the classroom.
Lesson 3: Post-training support is critical and must be planned and budgeted for early on. Even Getting the training up and running always seems like priority number one, but over the course of implementing the AGI pilots we learned that we needed to do a better job planning and budgeting for more structured and intensive post-training support from the very beginning of each project. The AGI pilots provided three to six months of post-training job placement assistance—such as internships, job search coaching, and so on—or business advisory services—such as business mentoring and check-ins, linkages to micro-franchises and business capital, etc. The exact balance of classroom training versus placement support hasn’t been rigorously tested, but our experience suggests this support can really help trainees put their new skills to use in the labor market. An extended follow-up period may be particularly important for young women just entering the labor market or breaking into non-traditional trades.
Lesson 4: Improving the monitoring and verification of employment outcomes is essential if we want to improve employment outcomes in skills training projects. Many projects don’t monitor attendance or performance during training, let alone keep track of participants after training ends. AGI pilots monitored business and job performance and verified employment outcomes up to six months after classroom training ended. The pilots relied on self-reporting by service providers, then verified these claims among a random sample of trainees (about 25 percent) by talking with employers, local women, and community members, and by accessing the trainee’s business records. The percentage of employed youth in the sample was then extrapolated to the population that the training provider claimed to be employed. In Liberia and Nepal, where pilots implemented results-based contracts, this extrapolation was used as a basis for the final payment. Any inaccurate claims by training providers proportionally reduced their payment and could jeopardize eligibility for future rounds of training. In the Resource Guide, you can download the employment/business verification strategy from the Liberia AGI, as well as tools for monitoring and placement verification.
Lesson 5: Performance-based incentives are operationally feasible—even in fragile settings—and seem to improve outcomes, though this is an area for more rigorous testing. We used results-based contracts for training providers in the relatively small program in Liberia, targeting 2,500 young women, as well as in the Nepal AGI, which was embedded in a larger program that trains 15,000 youth annually. Both projects achieved impressive results and we hypothesize that the performance incentives for the service providers accounts for this in part.
A forthcoming and final blog in this series will address recommendations for future learning and research from the AGI.