A key constraint to the growth of innovative SMEs in many developing countries is limited local demand. This may be particularly true in countries with small populations, such as those in the Western Balkans, where population sizes range from only 620,000 in Montenegro to slightly under 7 million in Serbia. A company developing cloud-based software for the energy sector, one developing digital tools for vineyard management, or another selling specialized safety equipment may all find that the number of potential customers in their home country is limited, and so their ability to grow and to demonstrate market traction to potential investors will depend on reaching customers in foreign markets. But it can be difficult for small and medium-sized innovative firms to overcome search frictions and reach foreign buyers.
In a new working paper, we report on a six-country randomized experiment with 225 innovative SMEs in the Western Balkans that tests the effectiveness of a capacity-building program called PowerUP in overcoming these barriers. These firms had a mean (median) of 17.6 (8) workers and were largely in the service sector with other firms as their main clients. The majority were exporting already, and the main challenge was how to acquire new customers and expand their exports, as well as bounce back from the short-term fall in demand that occurred during the start of the COVID-19 pandemic. Firms were randomized into two groups:
· A treatment group of 113 firms were offered 30 hours of virtual group-based training sessions over Zoom, as well as 5 one-on-one meetings with Deloitte consultants, who tailored the assistance to the specific needs of the firms. The training focused heavily on building customers and markets, including how to use digital marketing tools such as search engine optimization to help reach potential customers. Later sessions also covered HR, financial management and pitching. Initial take-up was high, with 97% attending some sessions, but dropped over time, with the median firm attending 6 out of 9 sessions and three one-on-one sessions.
· A control group of 112 firms were instead offered three 1.5 hour webinars that covered the same topics at a broad conceptual level, with attendance for these also dropping over time.
T The slide below shows an example of the treatment group homework assignments and practical sessions, focusing on how to grow their market, and actions like digitalization that they could try right away.
Measuring impacts and how to measure digital presence
One of the challenges with working with these types of firms is getting data on firm outcomes. Firms are spread out through six countries, and the owners of these firms are typically very busy and less willing to answer surveys than is the case with microenterprises. However, these types of innovative, relatively young, largely service-based firms also do not show up in administrative datasets – trying to convince the tax authorities of 6 countries to share data was impossible, and official export databases are much better at picking up goods than services, and also have a considerable lag in data availability.
We therefore relied on two sources of data directly collected from firms: (1) pitch decks and traction sheets that 60% of the firms provided 7 months after starting the program in order to have the chance to get scored by judges and be one of 11 firms selected to appear in a pitching live event; and (2) a follow-up survey that was conducted a year after the program started, in which, after much effort, we were able to get data on customers, exports, and other firm outcomes from 194/225 firms (88% of treatment and 84% of control firms).
Since the program focused so much on building customers and markets through digital tools, we also developed detailed measures of digital presence, which we considered one of the main intermediate outcomes of interest. To measure the extent to which firms were employing modern digital marketing methods, we hired two consultants to independently search the firms on Google and score their website and Facebook pages along several dimensions. They assessed the performance of firms on Google searches, reflecting the use of search engine optimization techniques. The consultants also scored the use of marketing tools and other relevant features of the website such as version in English or other relevant languages, loading time, responsiveness to mobile format, contact information and customer testimonials. In addition, they rated the quality of the Facebook page based on multiple dimensions such as the frequency of posts, promotions announcements, or media content appeal. Our scoring rubric is here.
The use of these measures in the evaluation has several advantages. First, it allows directly measuring a relevant outcome without relying on self-reported data by firms. Second, there is no attrition, for obvious reasons, allowing us to i) avoid any response bias concern for these outcomes and ii) alleviate response bias concerns regarding the survey responses instead, through a balance check for the attrition based on these outcomes. Thirdly, the data collection is relatively low cost and can be done remotely.
However, collecting this information involved overcoming a couple of measurement issues that we thought might be of interest to other researchers considering using this type of outcome for their evaluations, which is likely to become more important in the post-COVID era:
1) An obvious one was language, since many of the websites or social media pages were partially or exclusively in local languages: we hired consultants with linguistic knowledge of the two prevalent language families or languages across the six countries in the region – Serbo-Croatian and Albanian, and blinded them to treatment status.
2) A perhaps less obvious one was location. We were interested in the ability of customers in both overseas markets and in the Balkans to find firms. Although consultants were instructed to use incognito mode, Google still tailors search results to the location of the searcher up to a certain extent. Heterogeneous consultants’ locations – or the use of a VPN – can help capturing the performance of firms in different geographical contexts. The Google searches performance was the measure on which there was the most discordance between the assessments done by the two consultants. The correlation between the two consultants’ final indexes for the Facebook and website quality was over 80% (84% or the website and 83% for Facebook), while the correlation for the Google searches was quite lower, amounting to 68%.
3) Averaging out noise and differences in subjective rankings. The two consultants were trained on a sample of firms to ensure they understood the scoring task. Nevertheless, while most of the measures were objective (e.g., the firm showed up in the first page of Google search results, there is (or not) contact information, the Facebook page includes customer testimonials), there was some amount of subjective judgement required on some items. Any large discrepancies were discussed, and then scores were averaged.
Key results
We summarize three key results here, while a 2-page policy note provides some more details and policy recommendations:
1. Treated firms significantly improved their digital presence in a way that made it easier for potential international customers to find the firm and be attracted to their offerings: Using the digital presence measures discussed above, we find treatment caused firms to be 10 percentage points more likely to show up in the first page of Google search results, 15 percentage points more likely to have Facebook posts in English, 10 percentage points more likely to use special offers, and 6 percentage points more likely to include customer testimonials and stories.
2. We do not find any significant impacts on other aspects of management practices such as HR or financial management: this may reflect more drop-out of this component of the training, as well as the content being less directly actionable.
3. Treatment caused firms to significantly increase the number of customers and, conditional on exporting, to export more. Treated firms almost double the number of of customers and exports of the control group. However, the intervention had no impact on the extensive margin of exports. Some of the firms were in sectors were exporting was a lot more difficult (e.g. those providing healthcare, tourism and education services) and the program did not appear to help them as much.
Given our sample is only 225 firms, one concern is whether impacts on exporting are genuine or being driven by measurement error or outlier events that had nothing to do with the program. To address this concern, we coupled the econometric analysis with in-depth qualitative interviews. And we specifically targetted the interviews to firms that had improved their digital presence and expand exports in a significant way. These interviews were useful in confirming the observed improvements were genuine, and providing more detail on mechanisms. They suggest that the improvement in digital presence was largely a knowledge story, as pre-treatment firms did not know how to use these tools. The improvement in exports appears to come from a combination of sector- and market-specific advice offered by the Deloitte consultants on how to target the right customers in European markets and adjust product offerings, as well as an encouragement effect that helped firms unsure about expanding to other markets to be confident they were on the right track.
Join the Conversation