We know very little about what makes innovation policy work: Four areas for more learning

Photo Credit: Innovation Growth Lab.

Whether in Silicon Valley or Kenya’s furniture sector, innovation is a critical driver of job creation and economic growth. It could be a mobile app to connect farmers and buyers of agricultural products. Or perhaps an efficient and affordable solar roof tile. Innovation comes in many forms, from products and services to business models.

Yet despite the growing investment in policies to support innovation, we know surprisingly little about what makes these policies effective. To advance understanding of what works in innovation policy, Nesta, in collaboration with the Kauffman Foundation and the World Bank Group, organized the recent Innovation Growth Lab (IGL) Global Conference in London. The mission of IGL is to promote evidence-based innovation and entrepreneurship policies by funding randomized controlled trials (RCTs) and testing new policy approaches.
 
The conference was successful in discussing both research and policy challenges — a welcome change from typical innovation conferences, which often focus on either academia or policy.

From the event, I observe that there are still four important gaps in this field:
 

• Lack of evidence. There have been few impact evaluations of “narrow” innovation policy programs — that is, interventions that support the introduction of new products, new processes and technology adoption among firms — with comparison to the increasing body of RCTs in programs for supporting entrepreneurs. Despite the University of Manchester’s excellent work in compiling its compendium of evidence, we still know very little about what innovation policy instruments are effective.
• Lack of measurement of new instruments. Despite increasing efforts to pilot new instruments for innovation interventions, such as vouchers or public procurement, there has been little measurement of these new instruments or evaluation of alternative designs within the same instrument (for example what is the optimal co-financing level in a matching grant for innovation?). This lack of measurement is striking given the pilot nature of some of these programs, as well as the need to build evidence around how to best design these instruments to maximize their impact.
• Reluctance to implement RCTs. Many policy makers are still reluctant to implement RCTs in innovation programs. Their reason is that such programs are not suitable for randomization since they can only fund the best projects. Certainly, some innovation programs may be better evaluated using other methodologies. However, there is great need for increased use of RCTs and experimentation of new approaches. Furthermore, I am not confident that expert panels are capable of identifying the “best” innovation projects and evaluating the risks and likelihood of commercialization for these projects.
• Where there is evidence often there is a lack of consistent stocktaking. It is increasingly difficult to take stock of entrepreneurship and business education programs. While the number of RCTs in this area is rising, each trial tends to evaluate a different approach—for example, short courses versus longer modules, programs with or without mentors, varying business school approaches. Therefore, it is difficult to draw meaningful conclusions from these evaluations because similar programs are not replicated and measured across different contexts.

We still have a lot of work to do in developing evidence-backed innovation and entrepreneurship policies. Fortunately, there are a number of promising RCTs in the pipeline, such as an investment-readiness program in the Balkans from our colleagues Ana Cusolito and David McKenzie.

If you’d like to contribute to this effort, keep an eye out for the next IGL call for proposals on RCTs on innovation policy.