Let’s face it: assessing the results of a development project can be as complex as designing and implementing it. This is particularly true for projects that aim to create more and better jobs for all population groups and often work across sectors: how do we measure the number of newly created jobs through a private sector development project? Or the increase in earnings for young women and men who participated in a skills training, benefited from a coaching, or received stipends to help them move from low to higher quality jobs? Wouldn’t it be great to have a common terminology and definitions, and a set of ready-to-use tools to standardize the measurement of jobs outcomes?
Across the digital economy in Indonesia, both IT giants and smaller companies have the same complain: digital talents are hard to find. Obert Hoseanto, an Engagement Manager from Microsoft Indonesia, said the company recently contracted only five people for an internship program, out of a pool of hundreds of applicants.
But those applying for jobs are also struggling, with many realizing the difficulties of meeting the needs of their employers. Natali Ardianto is learning the ropes at tiket.com, a thriving start-up, “by doing”, he said. “Only 30% of the curriculum of my education was useful for the company I joined,” he explained.
A recent workshop held by the Coordinating Ministry of Economic Affairs and supported by the World Bank strived to develop a better understanding of this skills gap, by bringing in insights from the private sector, education experts, and global practitioners.
The disadvantages young women face in the labor market and in entrepreneurship in developing countries are not only substantial and complex, but they quickly compound. A plethora of forces drive gender disparities in youth employment: lack of opportunities to develop the skills demanded by the labor market, family or social pressure dissuading them from entering desirable jobs or male-dominated sectors, a detrimental work environment, or a lack of available services such as childcare might make achieving success an uphill battle. Yet innovative youth employment programs can respond to gender issues. Below are three examples presented in a recent virtual workshop held by the Solutions for Youth Employment (S4YE) coalition with members of its Impact Portfolio community.
Following milestones such as the World Humanitarian Summit, the momentum is strong for humanitarian and development communities to work together in complementary ways—not in sequence—to bridge the humanitarian-development divide. Development institutions are engaging much earlier than in the past, emphasizing the need to focus more on prevention and building resilience where they can play an active role.
Thanks to Information and Communication Technologies (ICT), we now have new ways of bridging the divide and integrating these two efforts. First, ICT platforms can bring development partners together to analyze, design, and track progress in a more unified and efficient way. They also offer an integrated system where multiple communication channels can operate at the same time. As a result, the notion of “continuous” development, whereby development experts pick up the work where humanitarian agencies left off, is progressively giving way to “contiguous” development, which offers humanitarian and development teams a chance to work more closely together.
The Adolescent Girls Initiative (AGI) through its eight pilots taught us a great deal about how to make skills training more female-friendly and support young women's transition to productive employment. In addition to all the lessons we learned about working with young women, the pilots also taught us a lot about how to improve the overall quality of skills training.
Our Top 5 quality enhancement lessons, along with links to more information, can be found online in our Resource Guide and here:
Lesson 1: Skills training projects need to set realistic expectations for self-employment versus wage employment. In contexts with limited opportunities for wage employment, skills training projects should help orient youth to the likelihood of self-employment and develop content suitable to different levels of aspiration in that sphere. In Liberia, for example, we offered a job skills track and a business skills track. We ended up having to gradually reduce the size of the job skills track from 35 percent of trainees in Round 1 to just 18 percent in Round 3 after our impact evaluation showed the employment rate in the business skills track was much stronger. This wasn’t easy—it involved changing the orientation of the client, the training providers, and the girls themselves.
Lesson 2: Involving the private sector can improve the market relevance and overall effectiveness of training. AGI pilots partnered with the private sector in the implementation arrangements by hiring private companies to provide training tailored to the needs of a specific firm/sector—as in the Rwanda AGI—and by hiring private sector training and employment service companies to deliver training and assist with job placement—as in Haiti, Liberia, and Nepal. We also took low-cost steps to engage the private sector throughout implementation. For example, the Liberia AGI organized Private Sector Working Groups to provide routine guidance on project activities and enlisted members of the private sector to inspire the trainees by serving as guest speakers in the classroom.
Lesson 3: Post-training support is critical and must be planned and budgeted for early on. Even Getting the training up and running always seems like priority number one, but over the course of implementing the AGI pilots we learned that we needed to do a better job planning and budgeting for more structured and intensive post-training support from the very beginning of each project. The AGI pilots provided three to six months of post-training job placement assistance—such as internships, job search coaching, and so on—or business advisory services—such as business mentoring and check-ins, linkages to micro-franchises and business capital, etc. The exact balance of classroom training versus placement support hasn’t been rigorously tested, but our experience suggests this support can really help trainees put their new skills to use in the labor market. An extended follow-up period may be particularly important for young women just entering the labor market or breaking into non-traditional trades.
Lesson 4: Improving the monitoring and verification of employment outcomes is essential if we want to improve employment outcomes in skills training projects. Many projects don’t monitor attendance or performance during training, let alone keep track of participants after training ends. AGI pilots monitored business and job performance and verified employment outcomes up to six months after classroom training ended. The pilots relied on self-reporting by service providers, then verified these claims among a random sample of trainees (about 25 percent) by talking with employers, local women, and community members, and by accessing the trainee’s business records. The percentage of employed youth in the sample was then extrapolated to the population that the training provider claimed to be employed. In Liberia and Nepal, where pilots implemented results-based contracts, this extrapolation was used as a basis for the final payment. Any inaccurate claims by training providers proportionally reduced their payment and could jeopardize eligibility for future rounds of training. In the Resource Guide, you can download the employment/business verification strategy from the Liberia AGI, as well as tools for monitoring and placement verification.
Lesson 5: Performance-based incentives are operationally feasible—even in fragile settings—and seem to improve outcomes, though this is an area for more rigorous testing. We used results-based contracts for training providers in the relatively small program in Liberia, targeting 2,500 young women, as well as in the Nepal AGI, which was embedded in a larger program that trains 15,000 youth annually. Both projects achieved impressive results and we hypothesize that the performance incentives for the service providers accounts for this in part.
A forthcoming and final blog in this series will address recommendations for future learning and research from the AGI.
From 2008-2015, we implemented pilots in eight countries, with the aim of supporting young women’s transition to productive employment. The AGI marked the Bank’s first experience working with this population—adolescent girls and young women—on this topic—skills and employment. We learned a great deal lot along the way, which we have collated in an online Resource Guide to share with other teams.
We tested two main program models—a classroom-based Technical and Vocational Education and Training (TVET) model that delivered job and business skills plus life skills, and a Girls’ Club model that delivered life skills and short livelihood trainings in community-based safe space clubs. Both significantly impacted economic outcomes for young women, though the Girls’ Club model was far less expensive.
In Liberia, for example, the Economic Empowerment of Adolescent Girls and Young Women (EPAG) project—a TVET, classroom-based program—increased participants' employment by 47 percent and earnings by 80 percent. In Uganda, meanwhile, the Girl's Club program raised the likelihood of girls’ engagement in income-generating activities by 35 percent and had large impacts on risky sexual behaviors and the girls’ experience of violence.
A recent meta-analysis of Active Labor Market Policies (ALMPs) suggests that average program effects tend to be larger for females, and that training and other human capital interventions are particularly effective among women. This suggests that both AGI models—classroom-based vocational training and community-based Girls' Club training—should continue to be implemented and tested across a variety of settings.
So what made the AGI projects effective for young women?
First, AGI worked hard to get girls into the projects and to keep them there—which is challenging in itself, especially for younger girls. We know that attrition from training programs is high, but we don’t really know the magnitude of the problem because so many projects don’t monitor or report individual attendance. When projects do report attrition and disaggregate by sex, they often find young women drop out more than young men and for different reasons. AGI pilots were able to successfully recruit young women and maintained completion rates above 90 percent.
Here are some of the steps AGIs took to recruit young women and retain them:
- Consulted directly with girls in a vulnerability assessment to identify their constraints and needs;
- Budgeted time and resources for specialized communications and recruitment campaigns targeted to reach girls;
- Held morning and evening sessions in safe, accessible locations and took steps to prevent and respond to cases of gender-based violence;
- Provided free on-site childcare so young mothers could participate;
- Provided access to mentors inside and outside the classroom who helped follow up when girls missed training;
- Incentivized good attendance with stipends tied to participation, small completion bonuses, and trainee commitment forms, and facilitated access to savings so that participants could safely retain their stipends and earnings; and
- Incentivized training providers to help girls transition to jobs through results-based financing schemes.
Providing girls with accurate information about the returns to various male- and female-dominated trades is a first step. An experiment in Kenya provided such information and was able to get young women to sign up for training in male-dominated trades, but later on they were no more likely to complete training or pursue work in those trades than young women who had not received the information.
Qualitative work in Uganda among women who successfully "crossed over" into male-dominated fields found the presence of male role models early in a young woman’s career was an important factor, suggesting further that information alone isn’t enough.
Here are some things AGIs did to break occupational segregation:
- Conducted local labor market assessments that intentionally explored market demand in non-traditional trades for women;
- Included an orientation period to educate participants about their training options;
- Encouraged women to enter non-traditional trades in groups;
- Supported participants with mentors, other role models from the community, and careful monitoring for potential unintended consequences.
So what’s the bottom line?
Being intentional about designing and implementing projects that work well for young women requires more planning and resources up front, but the results are impressive—making the investment worthwhile. A single program doesn’t have to do it all—strategies for making training female-friendly need to make sense in the local context.
Forthcoming blogs will explore broader "good practice" lessons from the AGI—not specific to young women—and highlight recommendations for future research and learning.
While Brazil faces a difficult fiscal and economic situation right now, I would like to view national progress on employment and incomes from a long-term perspective, which is valuable when addressing Education and Human Development issues in a broader sense.
Finding a job in Afghanistan is tough, and for young women even tougher. But with well-designed skills training and shifting attitudes, that may be changing.
In our last two blogs, we spoke about why measurement is key for development professionals and what should we measure? and about some take-aways from the medical profession on the measurement of competencies and performance. In this blog, we discuss specific ways we can use those lessons and apply them to the development sector.
As we discussed, in the medical world, lessons learned in competency and performance measurement relate to:
The focus on competencies, performance, and the space in between
Competence being specific to situations and existing on a continuum
Assessment as a program of activity that uses multi-source qualitative and quantitative information
The importance of the reproducibility of assessments
Encouraging the use of a portfolio.
But how can the above be specifically applied to development? Development practitioners can certainly take a page from the medical profession, as the stakes for getting measurement right are no less than bettering the lives of those who live on less than a dollar a day.