When I start discussing evaluations with government partners, and note the need for us to follow and survey over time a control group who did not get the program, one of the first questions I always get is “Won’t it be really hard to get them to respond?”. I often answer with reference to a couple of case examples from my own work, but now have a new answer courtesy of a new paper on testing for attrition bias in experiments by Dalia Ghanem, Sarojini Hirshleifer and Karen Ortiz-Becerra.
As part of the paper, they conduct a systematic review of field experiments with baseline data published in the top 5 economics journals plus the AEJ Applied, EJ, ReStat, and JDE over the years 2009 to 2015”, covering 84 journal articles. They note that attrition is a common problem, with 43% of these experiments having attrition rates over 15% and 68% having attrition rates over 5%. The paper then has discussion over what the appropriate tests should be to figure out whether this is a problem. But I wanted to highlight this panel from Figure 1 in their paper, which plots the absolute value of the difference in attrition rates by treatment and control. They note “64% have a differential rate that is less than 2 percentage points, and only 10% have a differential attrition rate that is greater than 5 percentage points.” That is, attrition rates aren’t much different for the control group.
Before we begin new posts next week, here are the 10 Development Impact posts published in 2018 that were most popular (by number of page views).
9. What’s new in education research? Impact evaluations and measurement – January 2018 round-up
8. What do we learn from increasing teacher salaries in Indonesia? More than the students did.
7. The State of Development Journals 2018: Quality, Acceptance Rates, Review Times, and Representation
6. What’s the latest in development economics research? Microsummaries of 150+ papers from NEUDC 2018
5. GiveDirectly Three-Year Impacts, Explained
Three from David McKenzie
The holidays are upon us. You might like to show off a bit by preparing something special for the ones you love. Why not make a pre-analysis plan this holiday season? You’re thinking, I do that every year, but we want to tell you about a new twist: using a dash of endline data!
- BITSS had its annual conference (program and live video for the different talks posted online). Lots of discussion of the latest in transparency and open science. Includes a replication exercise with all AEJ applied papers: “69 of 162 eligible replication attempts successfuly replicated the article's analysis 42.6%. A further 68 (42%) were at least partially successful. A total of 98 out of 303 (32.3%) relied on confidential or proprietary data, and were thus not reproducible by this project.” And slides by Evers and Moore that should cause you to question any analysis done using Poissons or Negative Binomials.
- development impact links
This is the twentieth in this year's series of posts by PhD students on the job market.
When we study how institutions affect development, we often focus on the characteristics of national institutions, such as whether a country is democratic, protects property rights, or has inclusive institutions. Yet villages in many developing countries contain almost no trace of these national institutions. Instead, life in rural villages is typically shaped by local leaders. In Sub-Saharan Africa, traditional leaders (namely village chiefs) are an important local institution. They control resources – most notably land – collect informal taxes, influence voting, and implement local development projects. The local importance of traditional leaders also concerns the nation-state. As national institutions attempt to increase their presence in the countryside, traditional leaders could act as complements or substitutes to state presence. They could either cooperate or compete with the public good provision by the state and thus enhance or weaken it.
In my job market paper, I study how local leaders and the national state interact. Specifically, I estimate the effect of state presence on the power, legitimacy, and effectiveness of village chiefs. In other words, do village chiefs become more or less influential when the national state is absent (or present) and how does this affect their public good provision? A key institutional feature in this context is that African states have used different strategies of dealing with traditional leaders that primarily vary on one dimension: whether chiefs are formally integrated into the state apparatus. I investigate how this institutional choice shapes the relationship between state presence and chiefs.
This is the nineteenth in this year's series of posts by PhD students on the job market.
While sociable peers increase your social skills, higher-achieving peers do not improve your academic performance. That is the main conclusion of my job market paper.
As the world bends closer towards automation, social skills take a lead role on individuals' well-being and labor market success. According to Deming (2017), between 1980 and 2012, jobs demanding high levels of social interaction grew by nearly 12 percentage points as a share of the U.S. labor force. Similarly, a recent column by the Washington Post highlights the importance of social skills for team productivity and employment opportunities. It describes the results of Google’s Project Aristotle, which concludes that the best teams at Google exhibit high levels of soft skills, and particularly social skills. These include emotional safety, equality, generosity, curiosity towards the ideas of your teammates, empathy, and emotional intelligence
While there is extensive research on policies that improve academic learning, little is known about how social skills form. My job market paper addresses this challenge. I present the results of a large-scale field experiment at boarding schools in Peru. The intervention was designed to estimate social and cognitive peer effects. While other studies have exploited random assignment to dormitories and classrooms, I use a novel experimental design to generate large variation in peer skills. Specifically, I assign students to two cross-randomized treatments in the allocation to beds in a dormitory: (1) less or more sociable peers, and (2) lower- or higher-achieving peers. This design surmounts many of the challenges with traditional approaches to the study of peer effects (Manski, 1993; Angrist, 2014; Caeyers and Fafchamps, 2016).
This is the eighteenth in this year's series of posts by PhD students on the job market.
In 2012, 700 million people in India suddenly found themselves without power for over 10 hours. At the time of the incident, political parties blamed each other for mismanagement and failing infrastructure. Such incidents reflect the extensive dysfunction in the sector, with technical problems and billing leakages that are among the worst in the world, amounting to 20% of electricity generated. The poor quality of electricity supply imposes major costs on the Indian economy; electricity shortages, for example, reduce manufacturing plant revenues by 5-10%. Why do these problems persist despite exponentially growing power generation? My job market paper shows that political corruption is one of the root causes behind unreliable electricity supply.
What is the link between political corruption and poor electricity supply? In democracies, incumbent politicians may consolidate power by favoring their voters with better access or lower prices. In India’s electricity sector, where politicians do not have direct control over electricity pricing, they may resort to illicit means in order to do this. Lower prices may actually benefit targeted consumers. But such patronage is costly: it hurts the revenues of electricity providers, inhibiting their ability to invest in infrastructure, and lowering electricity reliability for all consumers. While subsidies and increased access benefit consumers in targeted constituencies, the resulting underinvestment by providers may lead to unreliable supply.
Estimating the often-ambiguous welfare implications of corruption is, therefore, a challenge. Especially since detecting corruption is hard: corruption is frequently concealed, complicating the task of making causal inferences and identifying mechanisms of corruption. In this research, I develop novel methods to address these challenges, and find that political corruption in the electricity sector leads to large revenue losses for electricity providers, worsening their ability to reliably provide electricity.
This is the seventeenth in this year's series of posts by PhD students on the job market.
Aquinos, Bhuttos, Trudeaus, Yudhoyonos, Gandhis, Lees, Fujimoris: political dynasties remain ubiquitous in democratic countries. Though many societies democratised to end hereditary rule, nearly half of democratic countries have elected multiple heads of state from a single family. Politics is significantly more dynastic than other occupations in democratic societies. Individuals are, on average, five times more likely to enter an occupation their father was in. But having a politician father raises one's odds of entering politics by 110 times, more than double the dynastic bias of other elite occupations like medicine and law. Despite their prevalence and influence, we know little about the economic effects of political dynasties.
Effects of dynastic politics are theoretically ambiguous
Economic theory makes ambiguous predictions about how dynastic politics affects development. On the one hand, bequest motives might lengthen politicians’ time horizons and encourage them to make long-term investments. These founder effects could be good for economic development. However, if some political capital is heritable (e.g., a prominent name or a powerful network), dynastic politics may render elections less effective at selecting good leaders and disciplining them in office. These descendant effects are likely bad for development. The overall impact of dynastic politics is ambiguous, because it is the net result of founder and descendant effects.