Attend Spring Meetings on development topics from Apr 18-23. Comment and engage with experts. Calendar of Events

Syndicate content


Beyond ribbon-cutting: measuring the real impact of transport projects

Nancy Vandycke's picture
Photo: World Bank/Flickr
Development practitioners often rely on Monitoring and Evaluation (M&E) performance indicators to assess the results of a transport project. Collecting indicators before, during, and after a project allows us to gain insights about project execution and project outputs, which can help us, for example, measure changes in travel time or Bus Rapid Transit (BRT) system ridership. While this approach is important, well anchored into project design, and quite practical, it is not intended to evaluate “impact”. Observed changes in outcomes cannot be attributed to the project: many other external factors, such as economic conditions, interrelated policies or projects, or seasonal trends, also come into play. In other words, a descriptive approach fails to establish causality between a project or intervention and subsequent outcomes such as changes in income, labor markets, quality of life, or market efficiency.

To overcome the limitations of traditional M&E, the development community is increasingly turning to impact evaluation, an alternative approach whose methods more directly address the issue of causality. In that context, the World Bank’s transport experts have partnered with colleagues from the Development Impact Evaluation (DIME) team to rethink the way the impact of transport is measured. Two years ago, with support from the UK Department for International Development (DFID), a transport-dedicated impact evaluation program was launched: “IE Connect for Impact”. Now, impact evaluation is being implemented on 10 projects, covering rural roads, urban mobility, transport corridor development, and road safety. More projects will be selected toward the end of the year, as part of Phase II of the program.

The expected benefits are clear: informing project delivery during design and implementation, documenting the effects of policy and investment interventions, and prioritizing and filling knowledge gaps in the sector. Despite these significant benefits, transport accounts for less than 1% of all impact evaluation work —a very low proportion compared to the weight of other sectors such as in health (65% of all published impact evaluations), education (23%), agriculture and rural development (10%), or water (4%).

To measure the real impact of transport services, affordability needs to be part of the equation

Tatiana Peralta Quiros's picture

Differentiating between effective and nominal access

A couple of months ago, one of our urban development colleagues wrote about the gap between effective and nominal access to water infrastructure services. She explained that while many of the households in the study area were equipped with the infrastructure to supply clean water, a large number of them do not use it because of its price. She highlighted a “simple fact: it is not sufficient to have a service in your house, your yard, or your street. The service needs to work and you should be able to use it. If you can’t afford it or if features—such as design, location, or quality—prevent its use, you are not benefiting from that service.” To address this concern, the water practice has been developing ways to differentiate between “effective access” and “nominal access”—between having access to an infrastructure or service and being able to use it.

In transport, too, we have been exploring similar issues. In a series of blog posts on accessibility, we have looked at the way accessibility tools—the ability to quantify the opportunities that are accessible using a transit system—are reframing how we understand, evaluate, and plan transport systems. We have used this method that allows us to assess the effectiveness of public transport in connecting people to employment opportunities within a 60-minute commute.

Incorporating considerations of cost

Yet, time is not the only constraint that people face when using public transport systems. In Bogota, for example, the average percentage of monthly income that an individual spends on transport exceeds 20% for those in the lowest income group. In some parts of the city, this reaches up to 28%—well above the internationally acceptable level of affordability of 15%.

Habitat III will shape the future of cities. What will it mean for urban mobility?

Nancy Vandycke's picture
Photo credit: Rajarshi Mitra/Flickr

Next week, the international community will gather at Habitat III - the United Nations Conference on Housing and Sustainable Urban Development - to discuss important urban challenges as the world’s cities grow at an unprecedented rate.

Today, 54% of people live in cities and towns. Cities can be magnets for population growth and offer opportunities for jobs and social empowerment; but they can also be a source of congestion, exclusion and impoverishment. Which path of urban growth will prevail depends, in large part, on the quality and availability of mobility solutions. Transport is a structuring element of cities.

The reality of mobility in today’s cities is alarming— especially when measured against the four criteria that define sustainable mobility.

Empowering local women to build a more equitable future in Vietnam

Phuong Thi Minh Tran's picture

Vietnam’s economic emergence is perhaps best experienced along its rural roads: more than 175,000 kilometers of pavement, rubble and dirt track extend to two-thirds of the country’s population, including nearly all of the poorest people, who live among its productive farms, lush forests and meandering river valleys.

In recent years, road investments in Vietnam’s rural areas have improved socioeconomic development and promoted gender equity, social participation, improved school attendance, and more inclusive health services to impoverished regions. However, all but a few hundred communes remain off-grid, and infrastructural roadblocks and bureaucratic potholes have delayed the goal of a fully integrated road system.

The World Bank’s Third Rural Transport Project (RTP3) supported a win-win solution: employing ethnic minority women to sustainably manage road maintenance through an innovative participatory approach to local development. This blog entry describes the experience of improving the roads — and women’s lives — in rural Vietnam. Here are some of the lessons we’ve learned along the way:

Lesson 1: Solutions can come from unexpected sources.
The RTP3 task team’s investigation showed that up to a third of the population in Vietnam’s Northern Uplands provinces would be expected to contribute up to 10 percent of their total annual household expenditure to ensure safe passage along local roads — too much for most to afford. Furthermore, even when adequate resources are made available for maintenance, contractors have sometimes been unwilling to work in inaccessible regions for fear of mudslides during the rainy season.

How does accessibility re-frame our projects?

Tatiana Peralta Quiros's picture
The increasing availability of standardized transport data and computing power is allowing us to understand the spatial and network impacts of different transportation projects or policies. In January, we officially introduced the OpenTripPlannerAnalyst (OTPA) Accessibility Tool. This open-source web-based tool allows us to combine the spatial distribution of the city (for example, jobs or schools), the transportation network and an individual’s travel behavior to calculate the ease with which an individual can access opportunities.

Using the OTPA Accessibility tool, we are unlocking the potential of these data sets and analysis techniques for modeling block-level accessibility. This tool allows anyone to model the interplay of transportation and land use in a city, and the ability to design transportation services that more accurately address citizens’ needs – for instance, tailored services connecting the poor or the bottom 40 percent to strategic places of interest.

In just a couple of months, we have begun to explore the different uses of the tool, and how it can be utilized in an operational context to inform our projects.
Employment Accessibility Changes in Lima,
Metro Line 2. TTL: Georges Darido

Comparing transportation scenarios
The most obvious use of the tool is to compare the accessibility impacts of different transportation networks. The tool allows users to upload different transportation scenarios, and compare how the access to jobs changes in the different parts of the city. In Lima, Peru, we were able to compare the employment accessibility changes that were produced by adding a new metro line. It also helped us understand the network and connectivity impacts of the projects, rather than relying on only travel times.

Understanding spatial form
However, the tool’s uses are not limited to comparing transport scenarios. Combining the tool with earth observation data to identify the location of slums and social housing, we are to explore the spatial form of a city and the accessibility opportunities that are provided to a city’s most vulnerable population.  We did so in Buenos Aires, Argentina, were we combined LandScan data and outputs from the tool to understand the employment accessibility options available to the city’s poorest population groups.

The need to improve transport impact evaluations to better target the Bottom 40%

Julie Babinard's picture
In line with the World Bank’s overarching new goals to decrease extreme poverty to 3 % of the world's population by 2030 and to raise the income of the bottom 40% in every country, what can the transport sector do to provide development opportunities such as access to employment and services to the poorest?

Estimating the direct and indirect benefits of transport projects remains difficult. Only a handful of rigorous impact evaluations have been done as the methodologies are technically and financially demanding. There are also differences between the impact of rural and urban projects that need to be carefully anticipated and evaluated.

Can we simplify the methodologies?

Despite the Bank’s rich experience with transport development projects, it remains quite difficult to fully capture the direct and indirect effects of improved transport connectivity and mobility on poverty outcomes. There are many statistical problems that come with impact evaluation. Chief among them, surveys must be carefully designed to avoid some of the pitfalls that usually hinder the evaluation of transport projects (sample bias, timeline, direct vs. indirect effects, issues with control group selection, etc.).

Impact evaluation typically requires comparing groups that have similar characteristics but one is located in the area of a project (treatment group), therefore it is likely to be affected by the project implementation, while the other group is not (control group). Ideally, both groups must be randomly selected and sufficiently large to minimize sample bias. In the majority of road transport projects, the reality is that it is difficult to identify control groups to properly evaluate the direct and indirect impact of road transport improvements. Also, road projects take a long time to be implemented and it is difficult to monitor the effects for the duration of a project on both control and treatment groups. Statistical and econometric tools can be used to compensate for methodological shortcomings but they still require the use of significant resources and knowhow to be done in a systematic and successful manner.

São Paulo and Mumbai: Improving Mass Transit in Two BRIC Megacities

Jorge Rebelo's picture
Mumbai and São Paulo are two mega metropolitan regions (MMR and SPMR) in the BRICs with about 20 million inhabitants each. They are the economic engines of their respective countries and act as a magnet for rural, low-income populations seeking employment opportunities, growing at a rate that puts tremendous pressure on their transport infrastructure and other public utilities.

As population and income rise, car and motorcycle ownership quickly increased in both megacities while mass transit is not developing fast enough, with serious consequences on traffic congestion, accidents and pollution. São Paulo has 150km+ traffic queues daily and losses of productivity, wasted fuel, health impacts and accidents estimated at around 2% of Brazil’s GDP in 2013, with three fatal deaths daily in motorcycle accidents alone. Mumbai, in addition to all-day road traffic jams, have an astounding six deaths daily from riders hanging and falling from packed trains which circulate with open doors to avoid reducing carrying capacity. The city comes to a standstill when the rail right-of-way is flooded by heavy monsoon rains. 

Access to jobs and basic services in both mega-cities is extremely difficult – particularly for the poor, who often live far from major employment centers. The two cities need to act quickly and take drastic measures to improve mobility and access... But this is easier said than done: expanding the transport infrastructure in these megacities requires careful planning, massive investment,  and may also involve relocating large numbers of people and businesses.

Is Public Transport Affordable?

Julie Babinard's picture
When planning transport systems in developing countries, one of the main challenges is to evaluate the proportion of income spent by poorer households on transport as well as in understanding transport patterns in relation to residential location, travel distance and travel mode. High real estate prices in urban centers often force low-income households in developing countries to live farther out in the periphery, with consequences on the way urban agglomerations develop and with subsequent effects on the levels of motorization, congestion, local air pollution, physical activity and the expansion of urban poverty.

"¿Me lleva por 1000 pesos?" – Subsidios al transporte público para los pobres

Camila Rodriguez's picture
En nuestra revisión del proceso de modernización del transporte público en Bogotá, hemos leído una gran cantidad de análisis técnicos de los modelos de negocio, riesgos e incentivos en los contratos, temas operacionales, pero esta súplica por un pasaje reducido, "¿me lleva por 1000 pesos" ( la tarifa normal es de 1400 pesos) que aproximadamente el 23% de los pasajeros de los buses en Bogotá a veces le pide al co

“Will you take me for a 1,000 pesos?”— Making sure public transport subsidies really target the poor

Camila Rodriguez's picture
Also available in: Español

Follow the authors on Twitter: @TweetingCamila and @shomik_raj

A commuter in Bogotá, Colombia
(World Bank)
When analyzing the modernization of Bogota’s extensive bus system, we read a lot of technical analyses on public transport business models, risks, incentives, etc. But in a city where 11,6% of the population lives below the poverty line, social reality trumps all theoretical studies. In a new report entitled The promise and challenges of integrating public transportation in Bogotá, experts from Embarq estimate that as many as 23% of bus users sometimes plea for discounted rides, asking the bus driver: “Will you take me for 1,000 pesos?”

This situation points to one of the toughest challenges faced by public transport systems: how to reconcile financial sustainability and social inclusion? On the one hand, if fares do not cover operational costs, systems need subsidies to survive, which can pose serious financial and political risks. In some cases, transport systems operating with inadequate financial resources may experience system deterioration, safety problems and service curtailment. On the other hand, if fares are set to reflect the real price of transport services so that operators can recoup their costs without subsidies, then the poor are often priced out. This is exactly the problem that Bogotá is currently struggling with, despite its well-deserved reputation for innovation and excellence in public transport: fares of its state-of-the-art Transmilenio Bus Rapid Transit (BRT) system are pegged at cost-recovery levels that may price out many of the city’s poorest. Recent studies show that the lowest income households in Bogotá (socioeconomic strata 1, 2, 3 according to the local classification) are already devoting 20-30% of their total income to transportation, spending more than US$2 daily.  The situation may be even worse when Bogota’s city-wide public transit reform (the Integrated Public Transport System or SITP) is fully implemented, as bus drivers, with the adoption of smart cards, may no longer be able to offer unofficial discounts to passengers at their discretion, a common practice in the traditional system.