Conflict and violence are shrinking the space for development at a time when donors are scaling up their presence. To reconcile the conflicting objectives of staff safety with a need to do more (or a greater volume of investment), and doing it better (through higher quality projects), many development workers have started to rely on third party monitoring by outside agents, an approach that is costly and not always effective.The case of Mali demonstrates that alternatives exist.
Less than a decade ago Bank staff could travel freely around in Mali, even to the most remote communities in the country. But today, a mix of terrorism and armed violence renders field supervision of projects impossible in many locations.
To address this challenge—and in the wake of the 2013/14 security crisis in northern Mali—a monitoring system was designed that is light, low cost, and suited for monitoring in insecure areas, but also problem oriented and able to facilitate improvements in project implementation.
Led by World Bank staff with no stake in its outcome, this system focuses on a select set of issues and is iterative. The aim is to keep data collection focused, with few research questions and small samples. The data, once collected, is used to prepare reports quickly that draw attention to a few, critical issues.
By keeping the cost of data collection down, repeated data collection is feasible. This, in turn, acts as an additional incentive for project staff to take corrective action when such is needed.
Figure 1: Five steps, Iterative Beneficiary Monitoring (IBM) approach
An iterative feedback loop begins by getting intimately acquainted with a project. This is time consuming but indispensable if one is to collect relevant, high quality data. It also builds up trust with project staff and lays the groundwork for follow-up once results are produced.
Collecting information directly from beneficiaries is at the heart of the iterative feedback approach. After all, their experience on the project is what ultimately matters. In many instances, data samples do not need to be large. When standards or deadlines must be met (these are often defined in an Operations Manual), relatively few deviant data already pinpoint the problem.
For example, if all (or most) of 20 schools—randomly chosen but otherwise representative— report not receiving money for school feeding, there is clearly a problem.
It is important to keep the data collection concise; one has to resist the temptation to collect more than is strictly necessary. A project management team can only handle so many issues at any given time, and small samples and short questionnaires simplify enumerator training, reduce the cost of supervision, and make it possible for individual enumerators to travel on public transport or by motorbike to collect data, as opposed to entire data collection teams traveling in four-wheel drive vehicles.
Selectivity saves cost and minimizes exposure, which is critical when insecurity is an issue.
Collecting data by tablet and mobile phone
Even though the design phase is relatively time-consuming, a typical exercise can be designed in less than 4 weeks. Data can be collected using face-to-face interviews, but this tends to be costly and is not our instrument of choice. Where feasible, we prefer using mobile phone interviews as they are less expensive and avoid travel to insecure places.
In certain instances, we have also relied on enumerators from beneficiary communities who, once trained and equipped with tablets, report back to us regularly.
The data collected are analyzed and offered as feedback to project managers and Task Team Leaders (TTLs). Because there is little information, analysis is rapid and inexpensive. Reports are to the point and short, rarely more than 10 pages, and are discussed with the TTL and project staff.
The Bank’s management team receives a copy of the report, as do officials in charge of the project. With reports in hand, the project team can now take measures to address the issues identified.
Another round of data collection follows a few months later, with the aim of assessing improvements and, possibly, identifying new issues that might have arisen. The report follows the same route as the previous one. This is repeated on regular basis till the end of the project.
This approach, which we have labeled Iterative Beneficiary Monitoring (IBM), has now been applied to various projects in Mali. School feeding, fertilizer subsidies, and free medical care have all been monitored, and the results used to inform and improve operations. In all instances, the cost of data collection was less than $5,000 and the associated staff time, just a couple of weeks.
With this track record, it is unsurprising that there is considerable interest in scaling this method up. This is realistic. The core requirements for doing so are: a local Bank staff with diplomatic, data collection, and analytical skills, and a small budget to pay for data collection.
In comparison to the hundreds of thousands or even millions of dollars spent on third party monitoring, the agile and impactful approach developed in Mali seems worth replicating, both in fragile and other contexts.
Join the Conversation