Estimating the direct and indirect benefits of transport projects remains difficult. Only a handful of rigorous impact evaluations have been done as the methodologies are technically and financially demanding. There are also differences between the impact of rural and urban projects that need to be carefully anticipated and evaluated.
Can we simplify the methodologies?
Despite the Bank’s rich experience with transport development projects, it remains quite difficult to fully capture the direct and indirect effects of improved transport connectivity and mobility on poverty outcomes. There are many statistical problems that come with impact evaluation. Chief among them, surveys must be carefully designed to avoid some of the pitfalls that usually hinder the evaluation of transport projects (sample bias, timeline, direct vs. indirect effects, issues with control group selection, etc.).
Impact evaluation typically requires comparing groups that have similar characteristics but one is located in the area of a project (treatment group), therefore it is likely to be affected by the project implementation, while the other group is not (control group). Ideally, both groups must be randomly selected and sufficiently large to minimize sample bias. In the majority of road transport projects, the reality is that it is difficult to identify control groups to properly evaluate the direct and indirect impact of road transport improvements. Also, road projects take a long time to be implemented and it is difficult to monitor the effects for the duration of a project on both control and treatment groups. Statistical and econometric tools can be used to compensate for methodological shortcomings but they still require the use of significant resources and knowhow to be done in a systematic and successful manner.
To avoid methodological shortcomings, one option is to select a few key indicators to help track impact. In addition to relying on the Rural Access Index (a core World Bank indicator for the transport sector), some rural road transport projects have used traditional household surveys to capture data on key variables such as reduction in transport costs and time travelled to essential services (e.g. education, health, etc.), as well as access to employment opportunities.
Impact evaluations can also be supported by surveys purposely designed with indicators that are specific to the context of a particular project. For example, the evaluation designed to assess the impact of the Peru Rural Roads project showed significant reduction in travel times ( up to 50%), and a reduction in transport costs for both passenger and freight transport. There was also a substantial decrease in the percentage of time that the roads were not passable.
In Nigeria, the Lagos Urban Transport Project (FY02-FY11) provides another example of the collection of such data. The IBRD-funded project supported a Bus Rapid Transit (BRT) system, the first of its kind in Sub-Saharan Africa, and served as an example of a comprehensive and integrated approach to improving public transport. In Lagos, some 200,000 commuters are now using this bus system every day. In addition to safe, clean, and reliable transport, BRT passengers have also enjoyed a 30 percent decrease in average fares despite a twofold increase in fuel prices. Commuters have also cut their journey time by 73 percent, while the average waiting time has decreased by 33 percent. Time and money spent by poor households on travel was reduced from 90 minutes and Naira 150 in 2003, to 23 minutes and Naira 100 by June 2009.
Improving road impact evaluation can help better target the bottom 40%
When it comes to measuring the contribution of the transport sector to the Bank's twin goals, it will probably be easier to evaluate the impact of transport projects on the income of the bottom 40% ("shared prosperity" goal) rather than trying to assess the sector's ability to fight extreme poverty. This will mean first and foremost that we clearly identify who represents the two lowest income quintiles of the population (bottom 40%) in the targeted project area. This is because the characteristics of the bottom 40% will vary depending on the country and different geographical areas within each country.
We must now move forward to assess the impact of our sector interventions on these poverty levels more systematically and more accurately. In addition, relying on simpler data collection methods will not just serve the purpose of measuring important effects linked to transport improvements, it could also help prioritize investments better.
In urban transport, it is generally assumed that projects will influence land prices, which in turn will have an effect on housing prices and the choice of residential location of the poor. Improved transportation access might also increase the likelihood that individuals are employed, that they can access better jobs, reduce their travel time or improve their access to services. For rural transport, many countries still need the World Bank technical assistance in selecting the roads that need funding for repairs and maintenance. Without a simple methodology to help governments assess the need for transport investment in rural areas, poverty in these areas will remain critically high, and the rural poor will likely continue to migrate to urban areas in search of a better life.
Join the Conversation