Testing different behavioral approaches to get people to attend business training

A while back I blogged about work using active choice and enhanced active choice to get people to get flu shots and prescription refills. The basic idea here is that relatively small modifications to the way a choice is presented can have large impacts on the take-up of a program. This seemed useful in the context of many of our training programs– attendance rates averaged 65 percent in a review of business training programs I did with Chris Woodruff. Therefore for an ongoing evaluation of the GET AHEAD business training program in Kenya, we decided to test out this approach.
 

A  new paper in PLOS ONE, an open-access journal, presents the results (joint with Faizan Diwan, Grace Makana, and Silvia Paruzzolo). Our sample consists of 1172 female business owners who had been randomly selected to the treatment group in this study, and were to be invited to attend training. These are microenterprise owners mainly selling items in the market, with weekly profits averaging US$13.

Individuals were then randomly assigned to receive one of three types of invitations, which were read to them in Swahili by IPA project staff. All three invitations explained that the course was a 5 day workshop on business skills, and that the objective of this workshop is to “train you and build your capacity to operate your business or enable you to set up and operate a new business efficiently. This training typically costs 17,000 KSH (around US$200) to provide, and even though it is usually subsidized, most NGOs charge at least 2,000 KSH for it. We are pleased to offer it to you for free.”
The three different groups were then given the following different choice structures, which were explicitly structured to replicate the design used in the Keller et al. consumer psychology paper that I initially blogged about:
 

• Opt-in Group: please let me know if you will participate in the GET AHEAD training program
• Active Choice Group: Please let me know if
  • you will participate in the GET AHEAD training program
  • you will not participate in the GET AHEAD training program
• Enhanced Active Choice Group: Please let me know if
  • you will participate in the GET AHEAD training program in order to learn new business skills that could help you grow your business and take advantage of free training valued at KSH 17,000.
  • you will not participate in the GET AHEAD training program, and will turn down the opportunity to learn new skills for your business and choose not to receive KSH 17,000 of free training.

Note that the enhanced active choice explicitly highlights the benefits of participating (new skills to help grow the business, free training) and the costs of not participating (losing the opportunity to gain these new skills, and losing training valued at KSH 17,000).

Results
The figure below summarizes our main results for three different measures of attendance: saying they will attend, attending at least 1 day, and attending all 5 days of training (Keen readers will notice that I blogged about how to make this type of graph in Stata a while back). The good news was that we had pretty high attendance rates: 76% attended at least one day of training, and conditional on attending one day, 94.6% continued to finish all 5 days. However, the bad news from the point of view of hoping for results from our behavioral treatments is clearly seen in the figure: no significant differences across treatment groups. Moreover, the confidence intervals are pretty narrow, so these are reasonably precise zeroes.

Why didn’t this work?
Despite the promise offered by the active choice and enhanced active choice approaches, we find no significant effect on take-up in a business training program of using either approach compared to a more standard opt-in approach. We explore various reasons for the effect size being much less than that reported in Keller et al. We rule out lack of statistical power and also rule out that everyone who can attend is already doing so. We find suggestive evidence that the lack of effect is due to two things:
 

1. women being invited to training are not in fact the main decision-making agents in the decision of whether or not they attend: the opt-in choice has a larger effect for women who say they don’t need someone else’s permission to travel to another location, versus a smaller and even negative effect for those who need permission.
2. The enhanced active choice language may be too complicated for poor, relatively uneducated women: the enhanced active choice treatments have a 9 percentage point higher effect for individuals with more education or higher ability (on a Raven test) – but we can’t reject equality of treatment effects with education.

These last points are somewhat speculative, but point to the potential challenges of applying some decision-making insights from behavioral theory to developing country settings in which education levels are lower and decision-making power more dispersed than in the developed country settings for which these methods were developed. Hopefully others will experiment further with such approaches and report back on what they find.
For those interested, the baseline data and replication files are available in the World Bank open data library.