One popular solution to unemployment is to provide the unemployed with more skills through training. However, the impacts of vocational training in developed countries have been at most modest. There is much less well-identified work looking at impacts in developing countries, exceptions being randomized evaluations of programs in the Dominican Republic and Colombia. A new paper by Pushkar Maitra and Subha Mani adds to this evidence by reporting on an evaluation of a vocational training program for young women in one city in urban India.
Training details: the training was a 6 month course on stitching and tailoring services, provided by a collaboration between two NGOS – Pratham (well-known to impact evaluation readers for its work in education experiments in India) and a much smaller NGO called Social Awakening through Youth Action. Training was up to two hours per day, 5 days per week, and was offered in two low-income neighborhoods of New Delhi.
Participants: Participants had to be a woman aged 18 to 39, have at least 5 years education, and apply for the program. 658 Individuals applied, and two-thirds were randomized to the treatment group, and one-third to the control. At baseline the average participant was 22 years old, 51 percent were from Scheduled Castes, and only 5 percent had any form of employment. Participants had to pay 50 Rs per month to attend, and if they attended all 6 months of the program, got back 350 Rs. Fifty-five percent of those selected for treatment completed the program to get a certificate.
Follow-up survey: A single follow-up survey was conducted six months after the completion of the training, and was able to interview 504 of the 658 participants (76.6%). Attrition was slightly higher (26.4%) for control than for treatment (22.0%), although the attritors who answered the baseline survey (10% were not interviewed at baseline) do not look different on observables from the non-attritors. The authors inform me they have just finished a second follow-up 18 months post-program.
In addition, the authors try and measure a number of preference and behavioral characteristics such as risk aversion, competitiveness, and discount rates through lab games. Consistent with several other studies (e.g. Berge et al’s business training work in Tanzania), they have trouble getting this done for their full sample and only end up with data on this for 135 individuals, limiting how useful I view the part of the paper using this information.
So what do they find? Remembering these are 6 month post-training impacts, they find:
· The ITT impacts ignoring attrition show increases in employment, which are large in percentage terms relative to the very low base rate in the control group, but relatively modest in absolute terms: Any employment doubles from 6% in the control group to 12% in the treatment group; self-employment goes up 5.1 percentage points from the 1.2% rate in the control group.
· Monthly wage earnings also increase, going up 135 Rs ($2.50) relative to a base of 80 Rs, while self-employed earnings don’t change significantly.
· Treatment results in a higher likelihood of owning a sewing machine.
The total cost of the training to the NGO was 1810 Rs per person, so if these increases in earnings continue over time, the total cost of the program can be recouped in less than two years.
Issues and lessons
· How you measure employment matters: one would think that labor market evaluations should have an easier job knowing what their end outcome should be (employment) than programs with multiple end goals (e.g. CDD programs). However, as I have been finding in several evaluations I am working on, even “employment” is not necessarily well-defined. The authors look at 5 different measures (casual wage employment, full-time employment, self-employment, any employment, hours worked). They find no significant impact on full-time employment, but significant impacts on the other measures.
· How long do we need to see impacts? One big question is whether the impacts persist or not. In a context like this, where so few women work, and those that do work less than full-time, attachment to the labor market is not that strong. So these jobs created might be temporary. Alternatively, they may have just sped up how quickly people find jobs, so that the control group could catch up over time. An alternative leading to bias in the other direction may come from comparing the control group, who have had 12 months to find work, to the treatment group, who may have only 6 months to find work if they don’t look for work while training. Measuring impacts over multiple horizons seems important for these types of studies. The new 18 month follow-up data will thus be very interesting to see.
· Is attrition driving the results? The attrition rates of this study are not that much higher than those in the Dominican Republic and Colombian studies. But rates of 15-24% attrition are a big concern when the program effects one is looking at are only in the order of 5-6 percentage points – clearly any Manski-style bounding approach would lead to very wide and uninformative confidence intervals. Lee bounds or the bounding approach discussed several weeks ago here may help narrow these intervals while still accounting to some degree for attrition.
· Context, when is skills the constraint, and displacement: The authors note the context is one where India’s economy has been growing at 7 percent or so, meaning that employment opportunities are opening up but many youth lack the right skills to access them. But it is unclear to me that stitching and tailoring are really the skills where lots of jobs are likely to be opening up in a growing economy, and so a concern is that any employment found is just the result of displacement – those who got the training getting jobs that would have otherwise gone to other people.
Given the importance of employment as a policy issue worldwide, and the lack of impact evaluations for active labor market policies in many regions, studies like this are very welcome, and hopefully further work in this area can build on some of the lessons from this one. One point raised in correspondence with the authors is the difficulty in trying to obtain measures on attitudes and preferences like confidence etc – bringing people to a lab can be expensive and have high attrition, but while individual games to measure risk aversion and some other preferences can be done in the field, I don’t know of successful ways people have done games that involve others without having to bring people to a lab – anyone have experiences to share?
Join the Conversation