The typical role model intervention can be argued to be quite paternalistic and difficult to scale. For example, we think more women should be scientists and so have an inspirational woman scientist visit them. Or that more should study economics and so we have an economist come and tell them what a career in economics looks like. Such interventions can provide information and inspiration for young people who may otherwise lack exposure to these careers, but only provide exposure to one person and one career path. This may be appropriate in university when students have already at least opted into an introductory course and are being encouraged to continue on in the major. But what can be done at a high school level, where there are large numbers of students we want to reach, and students may have diverse career interests?
An at-scale experiment in Ecuador with multiple role models
In a new paper, published last week in Nature Human Behaviour, I teamed up with Igor Asanov, Thomas Åstebro, Guido Buenstorf, Bruno Crépon, Francisco P. Flores-Taipe, Mona Mensmann and Mathis Schulte to test an at-scale intervention that exposed Ecuadorian youth to both STEM and entrepreneurial role models. Our intervention takes place as part of a larger project of collaboration with the Ministry called ‘Showing Life Opportunities’, designed to promote learning in science and entrepreneurship.
We worked with a sample of 29,243 final year students (average age 17) in 813 Ecuadorian high schools. Half (407) of the schools were randomly assigned to the role model intervention, and the other half to a control group. In the treatment schools, students were assigned to watch two 30 minute videos, featuring randomly chosen segments from interviews with ten young (aged 24-34) entrepreneurs and ten young scientists from Ecuador, each containing a mix of both male and female role models. In these interviews the role models discuss what a typical day looks like in their work, why they chose this work, the skills needed for success, what inspires them, and their advice for those considering this career. Examples of STEM role models include an electrical engineer who had developed a device to help people with limited vision, an environmental scientist who had received a patent for an invention that uses crab waste for environmental remediation, and a molecular biologist working on cancer. Examples of entrepreneurial role models include business owners running companies in sustainable coffee, tourism and creative baby clothing. Students in the control schools instead watched regular educational content from the Ministry of Education’s EducaTV channel.
What happened?
A role model intervention was able to be delivered cheaply at scale with high fidelity: The online videos were able to be produced and delivered for under $1 per student, and take-up was high, with 94% of students watching between 19 and 20 of the 20 segments. This reflects efforts made to ensure internet access and the school system making this required content. (See this blog on another paper from this project on the impact of system-level interventions).
Showing multiple role models did change college major choice, but not in the way we had hoped: We use administrative records to track these students after graduation to see if they enroll in university, and if so, their choice of major. In total, 33% of students enroll in higher education, with this being slightly higher for girls (35%) than boys (32%). The role model treatment has a small, but not statistically significant, negative impact on the likelihood of continuing to higher education: -1.7 percentage points for girls, and -1.4 percentage points for boys.
The role model treatment caused both male and female students to be less likely to chose STEM as a major. STEM majors such as engineering and ICT are male-dominated, while business majors are female-dominated.
· Female students were 2.2 percentage points less likely to chose STEM as a major in the full sample and 5.0 percentage point less likely to chose STEM conditional on entering college. This further reduces the low representation of women in this field by 18% relative to the control mean. Instead they increase enrollment in business, making this even more female-dominated.
· Male students have a similar magnitude reduction in STEM majors of 3.4 percentage points unconditional on entering college and 4.9 percentage conditional on entering college although this is a smaller proportional reduction of 7% relative to the control mean. More of the male students switch to other majors apart from STEM or business, such as agriculture, and humanities and the arts.
We test, and find, no heterogeneity in these impacts by initial plans to study STEM, beliefs in their ability to be successful in STEM, or stereotypes about STEM. On average students saw a gender-balanced selection of role models, but we randomized at the individual level the proportion they saw of each gender. We do not find that outcomes differ for either males or females when seeing relatively more role models from their own gender.
Why did this happen?
We use surveys to measure changes in beliefs and attitudes. The journal Nature Human Behaviour has stricter standards than economics journals in what can be described as statistically significant – only results significant at the 5% level after adjusting for multiple hypothesis testing. By this standard, none of the impacts on beliefs and attitudes are statistically significant. The point estimates generally show more negative beliefs and attitudes towards STEM, and more favorable ones towards entrepreneurship. For example, in the subsample of women who continue to college, there is a 0.06 s.d. increase in agreement with stereotypes such as thinking that men have more natural aptitude in STEM subjects than women, or have more chance to succeed (p=0.027, q=0.064).
After seeing these results, we had a sample of 110 students watch the videos carefully and score them on 7 different dimensions: passion, charisma, how similar they thought the person was to them, the extent to which they thought the role model reinforced gender stereotypes, the extent to which they made the subject appear challenging, how much relevant information they provided, and stigma/weirdness.
The contrast between the two groups of role models seems to have mattered: students found the scientists made their subject seem more challenging and needing a lot more skills and study than entrepreneurs. This can reflect the reality of these careers. They also viewed the STEM role models as being weirder and having poorer social skills than entrepreneurs, as reinforcing stereotypes more, and having less passion and charisma.
What does this mean for role model interventions?
Our hope was that seeing interview segments with multiple role models of each type could provide students with a broader and more realistic view of the range of experiences in different career paths. Especially when delivering this nationwide, the idea was that students could see someone they could relate to, and that, at the high school level, it would stimulate considerations of emerging careers in STEM and high-growth entrepreneurship, where few children may have parents in these fields. However, seeing multiple role models may also lead to students getting less inspired by a single compelling and salient individual story, and instead to make comparisons of the commonalities and differences across them.
One of the big challenges of all of this work is that it is very difficult to say whether changes in what students study are welfare enhancing at either the individual or societal level. Under rational choice models of educational decision-making, providing a wider variety of role models should enable more informed and accurate decisions. Moreover, given how powerful role model effects can be, there is less risk of students being overly influenced by a single data point and potentially having false expectations around the ease of succeeding in a career. Ideally in future work we would set up an experiment which compared individual versus multiple career role models, and then tracked these students for at least a decade or more to see what leads to greater career success and happiness.
Bonus reading: My co-author Francisco offers a behind-the-paper post with some of the background, challenges, and lessons here.
Join the Conversation