Syndicate content


Tomayto tomahto: The research supply chain and the ethics of knowledge production

Humanity Journal's picture

Pre-test of Rural Household Survey, PakistanThis post is the first in a symposium from Humanity Journal on the changing nature of knowledge production in fragile states. It was written by Deval Desai, a Research Associate at ODI, and Rebecca Tapscott, a PhD Candidate at the Fletcher School at Tufts University.

Aid in the 21st century is increasingly evidence-driven. Between 2000 and 2006, the World Bank spent a total of $630 million on research. By 2011 the World Bank was spending $606 million per year, or about a quarter of its country budgets. In September of this year, by signing up to the Sustainable Development Goals, the global community enshrined a commitment to “increase significantly” a range of high-quality data over the next 15 years, to facilitate qualitative as well as quantitative understandings of growth and progress.

As the international community seeks to tackle the “hard problems” of development—fragility, conflict, endemic poverty—qualitative research is ever-more important. These problems are not amenable to best-practice solutions but must be tackled through deep contextual understanding of their drivers. Or so the policy story goes.[1] As a result, conducting qualitative research today is different from the days when Geertz set out for Bali. Gone are the intrepid individuals setting off to explore and explain an untouched environment, unaware of the demands of policymakers.[2]

We argue that while practice has changed, the ideology of qualitative research has not. Qualitative research is generally understood as the individual exercise of research methods to produce knowledge about the world, knowledge that can then be taken up by governance actors of all stripes. By contrast, we believe that today we must understand research as asystemic intervention, within the broader context of globalization and international development. Therefore, we should start with the political economy of contemporary research—an iterative, professionalized and increasingly saturated practice—to rethink the political and ethical implications of the research that we do.

As a first step to this end, we contrast two stylized frameworks for understanding qualitative research in fragile contexts: The “fragility research” framework, which we argue dominates the current debate; and the “research supply chain” framework, which we offer as a new framework and a provocation to discussion. We discuss each in turn, first considering how fragility research frames knowledge production in fragile or conflicted-affected states, identifying some assumptions the fragility research framework rests on, and critiquing some of its key conclusions. We then discuss the research supply chain as an alternative framework to explore the relationship between knowledge generation and policy. Finally, we raise some questions based on the new framework’s implications.

Who in this household has the final say?

Markus Goldstein's picture
Who in the household has decision making power over various things (kids going to school, health seeking behavior of individual members) either alone or jointly with someone else in the household makes up a set of questions that often find their way into surveys (e.g. a version is included in most Demographic and Health Surveys).  An interesting new paper by Amber Peterman and coauthors takes a hard look at these questions and what they might, or might not, be telling us.    

A curated list of our postings on Measurement and Survey Design

David McKenzie's picture
This list is a companion to our curated list on technical topics. It puts together our posts on issues of measurement, survey design, sampling, survey checks, managing survey teams, reducing attrition, and all the behind-the-scenes work needed to get the data needed for impact evaluations.

Have the MDGs affected developing country policies and spending? Findings of new 50 country study.

Duncan Green's picture

Portrait of childrenOne of the many baffling aspects of the post-2015/Sustainable Development Goal process is how little research there has been on the impact of their predecessor, the Millennium Development Goals. That may sound odd, given how often we hear ‘the MDGs are on/off track’ on poverty, health, education etc, but saying ‘the MDG for poverty reduction has been achieved five years ahead of schedule’ is not at all the same as saying ‘the MDGs caused that poverty reduction’ – a classic case of confusing correlation with causation.

So I gave heartfelt thanks when Columbia University’s Elham Seyedsayamdost got in touch after a previous whinge on this topic, and sent me her draft paper for UNDP which, as far as I know, is the first systematic attempt to look at the impact of the MDGs on national government policy. Here’s the abstract, with my commentary in brackets/italics. The full paper is here: MDG Assessment_ES, and Elham would welcome any feedback (es548[at]columbia[dot]edu):

"This study reviews post‐2005 national development strategies of fifty countries from diverse income groups, geographical locations, human development tiers, and ODA (official aid) levels to assess the extent to which national plans have tailored the Millennium Development Goals to their local contexts. Reviewing PRSPs and non‐PRSP national strategies, it presents a mixed picture." [so it’s about plans and policies, rather than what actually happened in terms of implementation, but it’s still way ahead of anything else I’ve seen]

‘Orderly traffic’ as a governance measure: a suggestion

Suvojit Chattopadhyay's picture

Traffic in IndiaMeasuring good governance can be tricky, but 'orderly traffic' can be used as an indicator since it offers insights beyond its limited definition.

As hard as it is to define ‘governance’, we have plenty of indicators to measure its quality: quality of key public services, extent of corruption, ease of doing business, etc. One of the challenges with these indicators is the distance between the process and outcomes, in particular, the assumptions involved in the translation of certain process into tangible outcomes. It follows that by mixing up indicators for processes and outcomes, we risk, well, measuring what doesn’t matter, and not measuring what does matter.

So as the title of this post suggests, could ‘orderly traffic’ be a good measure?

A familiar context: I live in Nairobi (and prior to that, in Delhi) and spend a considerable time waiting in traffic. What often makes traffic a problem is a complete lack of coordination amongst motorists on the road. However, I don’t think the onus of coordination at an intersection should rest on motorists – there are traffic lights or traffic police whose job it is to enforce discipline to ensure orderliness on the road. In many cities though, this is plain theory. In reality, traffic lights may not exist, or be broken; the traffic police may be absent, or just be incompetent. Motorists joust with each other every day and often end up creating gird-locks that hold everyone up. Please note that I am not talking about slow traffic caused purely due to long stops at intersections waiting for the lights to change. I am specifically concerned with the ‘orderliness’ of the flow. People waste time, fuel and a lot of their good humour (unless you are in a zen state) when they are in these gird-locks. It is usually more than evident to everyone whose fault it is and what the solution should be – and that usually only serves to raise tempers on the road. On days when the traffic flows smoothly, everyone seems happier. Zipping home after work is often the high point of the day.

Step-by-step: How to construct an emissions reporting system

Pauline Kennedy's picture
Oil and gas field. Asian Development Bank/Creative Commons

In preparing for a climate agreement in Paris, countries all over the world are planning their domestic strategies for cutting emissions. This often requires new policies to create incentives for low-carbon development, and for that, governments need accurate and comprehensive emissions data.

One important building block is a greenhouse gas reporting program, which a growing number of countries are working on. Mexico, for example, is gathering information from its newly established emissions reporting program to support its mitigation policies. The European Union’s and California’s reporting programs are essential to their emissions trading systems, and China’s reporting program will underpin its national trading system, planned for launch in late 2016.

At Carbon Expo today in Barcelona, the World Bank Group’s Partnership for Market Readiness with the World Resources Institute released the Guide for Designing Mandatory GHG Reporting Programs. Drawing on 13 existing and proposed greenhouse gas emissions reporting programs, the report looks at successful ways to build a strong data collection system and showcases best practices. It provides step-by-step guidance on developing and implementing these reporting programs.

Improving the Granularity of Nighttime Lights Satellite Imagery: Guest Post by Alexei Abrahams

Popular data
Nighttime lights satellite imagery (DMSP-NTL) are now a popular data source among economists. In a sentence, these imagery encompass almost all inhabited areas of the globe, and record the average quantity of light observed at each pixel (nominal size ~1km2) across cloud-free nights for every year, 1992-2012. In under-developed or conflicted regions, where survey or census data at a fine level of spatial and temporal disaggregation are seldom available or reliable or comparable over space or time, NTL and other satellite imagery can be an excellent resource. Recent economics papers have used NTL to study growth of cities in sub-Saharan Africa (Storeygard (2015)), production activity in blockaded Palestinian towns of the West Bank (Abrahams (2015), van der Weide et al (2015)), and urban form in China (Baum-Snow & Turner (2015)) and India (Harari (2015)).

Can you measure flows over short periods? Aka why Justin Wolfers might (NOT) want to reconsider that parenting study

David McKenzie's picture
Today in the Upshot, Justin Wolfers heavily criticizes a recent study that has received lots of media attention claiming that child outcomes are barely correlated with the time that parents spend with their children. He writes: