Socioemotional skills (also referred to as non-cognitive skills, character skills, or soft skills) have recently become part of the discourse on how to improve educational outcomes. There is growing evidence that those skills may be as important as intelligence in determining academic and professional success. There is already some evidence indicating that socioemotional skills can be encouraged.
Interventions aimed at fostering socioemotional skills continue to grow in number and scope. However, many questions remain about the extent to which these skills are malleable, exactly how they can be cultivated, and how to properly measure them. Available evidence comes from interventions with diverse goals, approaches, scales and contexts, making it impossible to generalize their results. There is still much to be learned in this field.
Organizations committed to cultivating socioemotional skills are making incremental advances evaluating their impact. The tools required to measure socioemotional skills for impact evaluation are in an early stage of development. Unlike some academic outcomes, socioemotional skills are not directly observable. As proxies, practitioners use scores based on self-report questionnaires that may be subject to biases. Therefore, researchers are developing alternative ways to measure socioemotional skills using tasks or games. A good example of those efforts is this study conducted in Turkey.
A crucial aspect in socioemotional learning is the role of teachers. Teacher qualities and practices may have different effects on students. A study in the U.S. found that the impacts of teachers on socioemotional learning and academic achievement are largely independent. What makes a teacher good at fostering academic achievement is not necessarily what makes him or her good at fostering socioemotional skills in students.
Evaluation of Enseña por México (Teach for Mexico)
Enseña por México is a member of Teach For All, a worldwide network of 47 independent organizations inspired by the approaches of Teach For America and Teach First (UK). Through an open call, every year Enseña por México recruits participants from a pool of recent college graduates with leadership potential to serve for two years as teachers in underperforming schools in Mexico. Participants are trained to be effective teachers and leaders. They are expected to make a difference in the classroom. In theory, by being exposed to participants, students are more likely to change their socioemotional skills in a positive way.
For our study, we focused on the short-term impact of Enseña por México. Treatment schools – the ones that received trained teachers – were jointly selected by educational authorities and Enseña por México based mainly on staff needs. Control schools – not impacted by the study – were chosen to match treatment school based on clear observable characteristics (number of students and teachers, academic performance). The sample includes 92 treatment schools and 236 control school. Both groups of schools were visited at the beginning and the end of the academic year 2016-17. We adopted a difference-in-differences statistical technique, comparing treatment and control schools at the beginning of the program (baseline) and at its completion (endline).
To estimate the impact of Enseña por México teachers on the socioemotional skills of their students, we applied questionnaires to over 25,000 students in grades 4-12. The responses provided scores for six types of socioemotional skills: self-management, growth mindset, self-efficacy, social awareness, grit, and academic locus of control. The results showed positive relation between scores in those six areas, and academic performance and good behavior. The responses also provided six measures of attitudes or behaviors related to socioemotional skills: educational expectations, perceived returns on education (general and pecuniary), tardiness and absenteeism in school, time devoted to homework, and community involvement. Those attitudes or behaviors can be thought of as more tangible expressions of underlying socioemotional skills.
We also measured teaching values and practices among the program participants and non-Enseña por México teachers using teacher questionnaires and a student perception survey.
Lessons Learned
Program participants are different from regular teachers in their teaching values and attitudes. They give more prominence to developing student curiosity (over competencies). They believe it is more important for students to set ambitious goals (over concrete and achievable ones). Participants engage in extracurricular activities, use evaluations, and frequently offer feedback to students. They also follow practices considered more conducive to better academic achievement and are perceived by students as more effective. Second, we find evidence of modest short-run improvements in grades 7-9 in self-management, growth mindset, self-efficacy and social awareness. We also find evidence of a reduction in tardiness and absenteeism in grades 4-12. We recognize that the main threat to the validity of the results is related to the selection of treatment schools. Control and treatment schools may not be entirely comparable. However, the magnitude and type of any potential bias arising from the selection process into end results are unclear. An important question is whether the results are driven by selection or by training of participants. That question can only be answered with an experiment designed for that specific purpose. Since training of participants is modest relative to the training and experience of regular teachers, we hypothesize that the observed effects are mostly due to selection. Lastly, it should be pointed out that cost-effectiveness calculations were not part of the scope of the project.
Applicability for Other Contexts
The evaluation provides an example of how socioemotional learning can be measured for impact evaluation purposes. It also illustrates how socioemotional scales are linked with behaviors and attitudes that lead to better academic achievement and higher educational attainment. The evaluation also addresses the crucial question of flexibility. The results suggest that socioemotional skills are adaptable: exposure to Enseña por México participants is associated with increases in some of the socioemotional skills analyzed.
The Teach For All network as well as other organizations and government programs may derive valuable lessons for their own agendas based on this evaluation effort. To our knowledge, this is the first large-scale impact evaluation focused on socioemotional skills in Mexico. Further research and evaluation is needed to better understand how the concept of “socioemotional learning” can be fostered more effectively. This study just scratches the surface.
The full report of the evaluation can be found in this link.
Follow World Bank Education team on Twitter @wbg_education.
Join the Conversation