Syndicate content

data collection

Data quality in research: what if we’re watering the garden while the house is on fire?

Michael M. Lokshin's picture

A colleague stopped me by the elevators while I was leaving the office.

“Do you know of any paper on (some complicated adjustment) of standard errors?”

I tried to remember, but nothing came to mind – “No, why do you need it?”

“A reviewer is asking for a correction.”

I mechanically took off my glasses and started to rub my eyes – “But it will make no difference. And even if it does, wouldn’t it be trivial compared to the other errors in your data?”

“Yes, I know. But I can’t control those other errors, so I’m doing my best I can, where I can.”

This happens again and again — how many times have I been in his shoes? In my previous life as an applied micro-economist, I was happily delegating control of data quality to “survey professionals” — national statistical offices or international organizations involved in data collection, without much interest in looking at the nitty-gritty details of how those data were collected. It was only after I got directly involved in survey work that I realized the extent to which data quality is affected by myriad extrinsic factors, from the technical (survey standards, protocols, methodology) to the practical (a surprise rainstorm, buggy software, broken equipment) to the contextual (the credentials and incentives of the interviewers, proper training and piloting), and a universe of other factors which are obvious to data producers but usually obscure and typically hidden from data users.

Project monitoring in fragile places does not have to be expensive

Andre Marie Taptue's picture



Conflict and violence are shrinking the space for development at a time when donors are scaling up their presence. To reconcile the conflicting objectives of staff safety with a need to do more (or a greater volume of investment), and doing it better (through higher quality projects), many development workers have started to rely on third party monitoring by outside agents, an approach that is costly and not always effective.
The case of Mali demonstrates that alternatives exist.

Less than a decade ago Bank staff could travel freely around in Mali, even to the most remote communities in the country. But today, a mix of terrorism and armed violence renders field supervision of projects impossible in many locations.

To address this challenge—and in the wake of the 2013/14 security crisis in northern Mali—a monitoring system was designed that is light, low cost, and suited for monitoring in insecure areas, but also problem oriented and able to facilitate improvements in project implementation.

Electronic versus paper-based data collection: reviewing the debate

This post was co-authored by Sacha Dray, Felipe Dunsch, and Marcus Holmlund.

Impact evaluation needs data, and often research teams collect this from scratch. Raw data fresh from the field is a bit like dirty laundry: it needs cleaning. Some stains are unavoidable – we all spill wine/sauce/coffee on ourselves from time to time, which is mildly frustrating but easily discarded as a fact of life, a random occurrence. But as these occurrences become regular we might begin to ask ourselves whether something is systematically wrong.

Issues of data collection and measurement

Berk Ozler's picture
About five years ago, soon after we started this blog, I wrote a blog post titled “Economists have experiments figured out. What’s next? (Hint: It’s Measurement)” Soon after the post, I had folks from IPA email me saying we should experiment with some important measurement issues, making use of IPA’s network of studies around the world.

A curated list of our postings on Measurement and Survey Design

David McKenzie's picture
This list is a companion to our curated list on technical topics. It puts together our posts on issues of measurement, survey design, sampling, survey checks, managing survey teams, reducing attrition, and all the behind-the-scenes work needed to get the data needed for impact evaluations. updated through October 23, 2018.
Measurement

Taming the Terra Incognita of PPPs: The case for data as an exploration tool

Fernanda Ruiz Nunez's picture
Image courtesy of history-map.com
The PPP territory spans the globe, and the debate over its effectiveness as a financing tool to achieve development goals reaches equally far and wide.

​Most recently, the Financing for Development Conference in Addis Ababa, Ethiopia sparked even more discussion about the role of public-private partnerships. The official line, spoken in a multitude of tongues, is that PPPs have an important role to play, and results are dependent on projects being procured, managed and regulated well. But one thing is clear in every language: “results” are based mainly on anecdotal evidence and case studies where attribution remains dubious, and findings cannot be generalized as they depend on the particular characteristics of the specific projects.
 
We can do better. As economists, development professionals, finance experts, and explorers of new and creative solutions to solve the problem of poverty, we must do better. And we will – with better data.
 
Lack of data has constrained the empirical literature on PPPs, in turn constraining our ability to tap the territory of PPPs and its potential to transform markets. After all, what do we really know about the economic impact of PPPs? Our first-ever literature review, underway now (the first draft is available at https://www.pppknowledgelab.org/ppp_document/2384), has laid an initial foundation for knowledge, and we have made the first draft available so that colleagues and interested individuals and organizations can contribute their data.

Can our parents collect reliable and timely price data?

Nada Hamadeh's picture

During the past few years, interest in high-frequency price data has grown steadily.  Recent major economic events - including the food crisis and the energy price surge – have increased the need for timely high-frequency data, openly available to all users.  Standard survey methods lag behind in meeting this demand, due to the high cost of collecting detailed sub-national data, the time delay usually associated with publishing the results, and the limitations to publishing detailed data. For example, although national consumer price indices (CPIs) are published on a monthly basis in most countries, national statistical offices do not release the underlying price data.

 
Crowd sourced price data

The many faces of corruption: The importance of digging deeper

Francesca Recanatini's picture

About a month ago two colleagues (Greg Kisunko and Steve Knack) posted a blog on “The many faces of corruption in the Russian Federation”. Their post, based on the elegant analysis of the 2011/2012 Russian BEEPS, underscores a point that many practitioners and researchers are now beginning to appreciate because of the availability of new, disaggregated data: corruption is not a homogenous phenomenon, but rather a term that encompasses many diverse phenomena that can have profoundly different impact on the growth and the development of a country. If we delve deeper into this disaggregated data, we observe that within the same country can coexist significantly different sub-national realities when it comes to the phenomenon we label “corruption”.

What can marketing experiments teach us about doing development research?

David McKenzie's picture

The March 2011 issue of the Harvard Business Review has “a step-by-step guide to smart business experiments” by Eric Anderson and Duncan Simester, two marketing professors who have done a number of experiments with large firms in the U.S. Their bottom line message for businesses is:


Pages