This is the first in our series of posts by Ph.D. students on the job market this year.
One of the key challenges of markets is to assess the quality of goods. A look at online dating websites – a market where information asymmetries loom particularly large - shows different ways in which people try to communicate that they are of “high quality”. A common strategy is to start your introduction with “My friends describe me as…” (to be followed by some glowing testimony “…smart, athletic, high-achieving – yet humble”). Why may this strategy not be effective? It raises questions about whether these friends are truthful and whether they have all the relevant information about your quality as a partner. The really interesting question you never see answered is: “How would your ex-partner describe you?”
My job market paper “The Value of Reference Letters”, coauthored with Rulof Burger (SU) and Patrizio Piraino (UCT), is about the challenges hiring firms face in identifying high-quality applicants. While the literature has largely focused on the role of friends and family members (Topa 2011, Beaman and Magruder 2012) in job referrals, we investigate whether information from ex-employers can facilitate the matching process. Specifically, we test the effect of a standardized reference letter asking previous employers to rate workers on a range of hard skills (e.g. numeracy, literacy) and soft skills (e.g. reliability, team ability).
In many developing countries, firms struggle to identify capable candidates given their lack of expertise in screening and the fact that educational credentials are perceived to be weak signals of productivity. As a result, a majority of firms hires through informal channels. Why could this be a problem? For one, it may adversely affect match quality as it limits the pool of potential applicants (Rees 1966). Second, it exacerbates (gender) inequity: in South Africa and many other countries women have less access to informal referral networks (Magruder 2010, Beaman et al. 2013).
We conducted a series of field experiments with unemployed youth in townships surrounding Johannesburg, South Africa to answer three questions: 1) Are reference letters valuable? 2) Why are they valuable? 3) Why are they not more widely adopted?
Are Reference Letters Valuable?
First, we encourage 430 job seekers to ask a former employer to complete our reference letter template. We submit applications for the 31% of participants who return the completed letter; specifically, we send applications to online vacancies with and without the reference letters and compare firm responses. We find that the reference letters increase firm responses by about 60% (from 4.2 to 6.5%). Women, traditionally excluded from many referral networks, particularly benefit: reference letters increase firm responses by 89% (although this gender difference is not statistically significant).
Example of the standardized template
While letters are valuable when submitted by us, their effectiveness may differ when job seekers use them in their search. To see whether this is the case, we conduct a second field experiment with a sample of 1,267 job seekers, in which we encourage half of the participants to obtain a letter and subsequently follow their job search behavior and employment outcomes. After three months we find similar results: men are not more likely to find jobs, but employment rates for women who have reference letters double (from a base of 11.7%), thus fully closing the employment gender gap in our sample.
Why Are Reference Letters Effective?
To understand why reference letters are effective, we then test several additional questions. For one, do reference letters actually contain information that is both truthful and novel? We find that “subjective” employer ratings for numeracy and literacy are highly correlated with “objective” results from an aptitude test (for both female and male job seekers), even after we control for characteristics like age, education and school grades that are easily observable to firms.
We then test whether firms actually use this information or whether they just respond to the fact that a job seeker is able to obtain and correctly use a reference letter. While we find that more positive letters are more effective for getting job interviews, letters in which the former employer gives the highest rating in every category are completely ineffective and may even reduce chances of getting an interview (despite the fact that job seekers with these glowing reference letters perform very well in the aptitude test). This highlights the importance of signal credibility. Interestingly, the content of letters from female job seekers matters more, suggesting that firms are more uncertain about their ability.
Do reference letters lead to an overall increase in employment or are we just reshuffling the queue of job seekers? We find that attaching a letter increases the correlation between job seekers’ aptitude and firm responses by a factor of four, suggesting that firms are able to identify more productive candidates. The ability to hire more capable applicants may induce firms to increase their demand for workers (Mortensen Pissarides 1994). However, the study design does not allow to conclusively test for these (general equilibrium) effects.
Why are Reference Letters not more widely adopted?
In light of these results, one might ask why reference letters are not more prevalent. Data suggests that it is not due to the unwillingness of previous employers to provide letters: among participants that approach a former employer, 73.6% succeed in having the template completed. If former employers are willing to write letters and hiring firms use the information in the screening, then the “culprit” must be the job seeker.
We conduct a third experiment that tests two potential explanations for why job seekers do not ask for reference letters: they may underestimate their potential value or they may procrastinate in getting letters. After job seekers visited the job center they are randomly assigned to receive a i) reminder to return the letter, ii) a monetary incentive (half a daily wage) or iii) information on the effectiveness of letters. We find that the information treatment increases take-up (by about 65%) while the monetary incentive has no effect, suggesting that job seekers underestimate the potential value of reference letters.
Open Questions
While our study looks at the effects of reference letter in a static framework, reducing information asymmetries may also have dynamic effects. In ongoing work, I explore whether workers exert more effort on the job when they know they receive a reference letter. This could boost worker productivity beyond improvements in job matching explored in this study.
Despite these findings, it remains an open question whether reference letters improve welfare; high ability job seekers without previous work experience may struggle to get a foot in the door as hiring managers prefer applicants with letters. Reference letters may become an entry barrier and thus reduce market efficiency (Pallais 2014).
So does your ex know better than your friends? As economists we believe that there must be reason why in many countries (including South Africa) the majority of workers are hired through informal networks rather than formal hiring channels. Our study identifies market frictions that could explain why the common hiring practice may be sub-optimal. A simple intervention reducing information frictions shows promising results. Yet, the jury is still out. In ongoing work, I hope to shed light on this important question by comparing the performance of workers hired through different channels.
Martin Abel is a PhD Student at Harvard University. You can find his website here.
Join the Conversation