But how do we measure what soft skills youth have? One approach has been to rely on a range of survey questions. Examples from a range of different surveys fielded in the US are given here , and include Likert scales for questions like “I can work with someone who has different opinions than mine” and “I keep going with work even when it takes longer than I thought it would”. The World Bank STEP skills measurement  exercise employs such an approach in multiple countries, measuring personality traits, grit, and behavior skills.
However, one might be concerned about the subjectivity of some of these reports, and the extent to which people give socially desirable answers. The question then is whether one can have more objective measures of key soft skills.
In a short paper  I am presenting today at the ABCDE conference , Matthew Groh, Tara Vishwanath and I discuss the use of psychometric testing to reduce labor market asymmetries in Jordan. We worked with Dr Marwan Al-Zoubi, a Jordanian psychology professor to design a number of tests for potential job seekers. One set of these tests were for soft skills measurement.
Soft skills were measured by three interactive exercises:
- Group Exercise: The group discussion was centered on the design of an amusement park. Five to eight participants were tasked to redesign a failing amusement park in Jordan. Each participant was given a pre-defined role in marketing, HR, finance, customer service, and management with specific responsibilities and often conflicting goals. By creating an opportunity for job candidates to interact in a group in a structured manner, evaluators could evaluate how job candidates work in groups. Two trained evaluators evaluate each candidate on 10 soft skill categories during the group discussion, tabulated below:
2. Role-playing exercise: a one on one exercise with the evaluator and the job candidate that is intended to test the candidate under pressure. The job candidate plays the role of a customer service associate and the evaluator plays the role of an angry customer who just bought a computer that broke down the past night. The job candidate has to calm down the customer and come up with a solution within the framework of the companies’ rules. This component is assessed by one evaluator on soft skill components 1 through 5 in the table above.
3. Skills-based interview: where the evaluator asks questions to elicit examples of leadership, teamwork, and overcoming obstacles. This last component was assessed by one evaluator on the soft skill components 6 through 10 in the table above.The group exercise lasted 40 minutes, and the individual exercises lasted 5 to 10 minutes each. We then average the scores in each of the 10 soft skill categories and form a principal component of the average scores in the 10 soft skill categories to create a soft skills index.
We did this for 1567 recent graduates of universities and community colleges in Amman, Jordan. 58 percent of the sample were female, average age was 23, and the most common majors were accounting and business, engineering, and computing and IT.
Does this predict employment?
A follow-up survey that took place on average 10 months after the soft skill assessment suggests that this exercise is capturing something that matters to employers.
- For females, employment was 41.7% for those in the bottom quartile of soft skills, versus 55.3% for those in the top quartile.
- For males, employment was 54.5% for those in the bottom quartile, compared to 64.7% for those in the top quartile.
- Conditional on being employed, both males and females earn approximately 30 percent more per month if their soft skills are in the top quartile vs the bottom quartile.
So it seems that this measure is picking up something that both seems reasonable in spirit, and seems to have useful predictive power for employment in practice.
When will this be most useful?
Clearly this is a non-trivial measurement exercise, taking a full hour per person, which is as long or longer than many surveys. It requires trained staff to do the assessments, and requires bringing several people together in a group for the group assessment. So it is not an easy substitute for a set of self-reported questions.
It therefore seems most useful for the following contexts:
- Interventions that teach soft skills training – ideally one would like to measure whether these programs actually improve soft skills, before then tracing their impacts on ultimate outcomes like employment.
- As a way of measuring soft skills to help in matching workers to jobs – our ongoing work in Jordan looks at an intervention which does this.
- In contexts like some youth programs where you have people coming into workshops or several day sessions anyway, in which the logistics may be easier.