Can you help some firms without hurting others? Yes, in a new Kenyan business training evaluation

There are a multitude of government programs that directly try to help particular firms to grow. Business training is one of the most common forms of such support. A key concern when thinking about the impacts of such programs is whether any gains to participating firms come at the expense of their market competitors. E.g. perhaps you train some businesses to market their products slightly better, causing customers to abandon their competitors and simply reallocate which businesses sell the product. This reallocation can still be economically beneficial if it improves allocative efficiency, but failure to account for the losses to untrained firms would cause you to overestimate the overall program impact. This is a problem for most impact evaluations, which randomize at the individual level which firms get to participate in a program.

In a new working paper, I report on a business training experiment I ran with the ILO in Kenya, which was designed to measure these spillovers. We find over a three-year period that trained firms are able to sell more, without their competitors selling less – by diversifying the set of products they produce and building underdeveloped markets.

The Intervention
The training program is the ILO’s Gender and Entrepreneurship Together (GET-Ahead) program. This is a participatory program specifically designed for low-income women running small businesses, and covers both standard business training concepts (developing business ideas, marketing, financing, bookkeeping), as well as topics designed to help women overcome other challenges they face in running businesses (promoting the idea of gender equality and discussing the difference between the few biological differences in ability to perform tasks by gender versus the perceived gender attitudes towards this ability; how to divide household and business tasks; how to network with other women, etc.), as well as attempting to generate a business mindset. The program is a full-day course for 5-days, and cost between $222 and $333 per woman trained, but was offered to them for free. Take-up was 77.7%.
We also later gave half the sample who had received this training a mentoring intervention, in which they met in a small group with a mentor every two weeks for 5 months, along with a monthly one-on-one meeting. This additional service cost $553 per women trained.

Study population and Experimental Design
Our sample consists of 3,537 women in 157 markets in four counties of Kenya. These markets are typically small and remote, largely consisting of women operating a limited variety of businesses such as selling fruits, vegetables, grains, and dried fish products from tables; and offering services like hairdressing, dressmaking and small food kiosks.
We then conducted a two-stage randomization: First, markets were assigned to treatment (93 markets) or control markets (64 markets). Then within the treatment markets, we formed strata based on baseline profits, and randomly assigned half the individuals in each sample to be invited to training. This results in a treatment group of 1172 women, a spillover control group of 988 women in the same markets, and a pure control group of 1377 women in the untreated markets.

Results
The figure below shows one of the key results in the paper. After three years, the treatment group’s profits were 221 KSH per week higher than the 1439 KSH earned by firms in the pure control group (p=0.014), a 15 percent increase. The spillover control’s profits are within 2 percent of the pure controls, and not significantly different (p=0.712).


When we examine impacts at the market level, the total sales volume in the treated markets has increased, as has the number of customers. When we examine mechanisms, the treated women have diversified the availability of goods in the markets in at least two ways. First, they keep more regular business hours, being more likely to open at a set time every day – so customers may be able to find goods at times they previously could not. Second, they have diversified their product offerings, selling products that they weren’t previously offering, so customers have a wider variety of goods to choose from in the markets. As a result, the underdeveloped market offers more to customers, who now buy more.

The training program therefore showed significant impacts, without harming competitors. In contrast, we can’t reject that mentoring had no additional effect. The training seems like it passes a cost-benefit test, but the mentoring does not.

Some more specific points of interest for impact evaluation

• Measurement of key outcomes is always a challenge. One thing we tried here was taking photographs of the inventories of the businesses, and then having independent enumerators count and value the goods, to see if we could physically observe changes in the business over time. I’ll blog more about this in another post, but the photo-value of inventories was larger in the treated firms (although not significantly so), and a simple comparison of whether the business looked bigger at follow-up than baseline did show significant treatment impacts.
• We did several things to improve statistical power here:
  • We worked with a much larger sample than typical training evaluations, and then imposed a screening criteria to make the businesses more homogenous to start with.
  • Following my more T approach, we did a long survey, followed by a shorter survey a month or two later to recollect profits and sales data, then use these multiple measures to give more power.
• The one-year results were not as large or significant as the three year results. There was also a suggestion of negative spillovers over this time frame. We were therefore afraid the training hadn’t been enough, and implemented the mentoring as a response, hoping to get clarity on this spillover issue. But it appears that firm owners did not get more financing after training, but slowly reinvested additional profits to build their inventories and product mix, taking time for the effects to show – and over a longer period, no evidence of spillovers is seen.
• We also measured broader life outcomes: not only are the treated women more profitable, but they have improved mental health, a higher subjective standard of living, and are more optimistic about their future well-being.
• As well as continuing to track outcomes for the firms surveyed at baseline, we did new market censuses to see whether market entry was affected by the intervention (we find no impact on this measure). I haven’t seen this measured in prior studies.
• Chris Woodruff and I wrote a review piece on what prior business training experiments had found. I have talked to quite a few people who have interpreted that paper as saying that business training has no effect. But what we pointed out in that paper was that most training experiments were very underpowered, working with small and heterogeneous samples. The treatment impact found in this Kenya experiment is no larger than the point estimates from a lot of these previous studies, it is just that we can measure it a lot more precisely, and can rule out that it is coming at the expense of competitors.