Syndicate content

David McKenzie's blog

What does a game-theoretic model with belief-dependent preferences teach us about how to randomize?

David McKenzie's picture

The June 2017 issue of the Economic Journal has a paper entitled “Assignment procedure biases in randomized policy experiments” (ungated version). The abstract summarizes the claim of the paper:
“We analyse theoretically encouragement and resentful demoralisation in RCTs and show that these might be rooted in the same behavioural trait –people’s propensity to act reciprocally. When people are motivated by reciprocity, the choice of assignment procedure influences the RCTs’ findings. We show that even credible and explicit randomisation procedures do not guarantee an unbiased prediction of the impact of policy interventions; however, they minimise any bias relative to other less transparent assignment procedures.”

Of particular interest to our readers might be the conclusion “Finally, we have shown that the assignment procedure bias is minimised by public randomisation. If possible, public lotteries should be used to allocated subjects into the two groups”

Given this recommendation, I thought it worth discussing how they get to this conclusion, and whether I agree that public randomization will minimize such bias.

Weekly links July 7: Making Jakarta Traffic Worse, Patient Kids and Hungry Judges, Competing for Brides by Pushing up Home Prices, and More…

David McKenzie's picture
  • In this week’s Science, Rema Hanna, Gabriel Kreindler, and Ben Olken look what happened when Jakarta abruptly ended HOV rules – showing how traffic got worse for everyone. Nice example of using Google traffic data – MIT news has a summary and discussion of how the research took place : “The key thing we did is to start collecting traffic data immediately,” Hanna explains. “Within 48 hours of the policy announcement, we were regularly having our computers check Google Maps every 10 minutes to check current traffic speeds on several roads in Jakarta. ... By starting so quickly we were able to capture real-time traffic conditions while the HOV policy was still in effect. We then compared the changes in traffic before and after the policy change.”All told, the impact of changing the HOV policy was highly significant. After the HOV policy was abandoned, the average speed of Jakarta’s rush hour traffic declined from about 17 to 12 miles per hour in the mornings, and from about 13 to 7 miles per hour in the evenings”
  • From NPR’s Goats and Soda: 4-year kids of Cameroonian subsistence farmers take the marshmallow test, as do German kids – who do you think did best?

Weekly links June 30: 7th grade development economics, the beginning at the end approach, stuff that happened a long time ago still impacts today, and more…

David McKenzie's picture
  • How to teach development economics in 20 minutes to 7th graders – Dave Evans explains his method.
  • The “beginning at the end” approach to experimentation – written from the point of view of business start-ups, but could easily apply to policy experiment work too “The typical approach to research is to start with a problem. In business, this often leads to identifying a lot of vague unknowns—a “broad area of ignorance” as Andreasen calls it—and leaves a loosely defined goal of simply reducing ignorance…“Beginning at the end” means that you determine what decision you’ll make when you know the results of your research, first, and let that dictate what data you need to collect and what your results need to look like in order to make that decision.”

Should we require balance t-tests of baseline observables in randomized experiments?

David McKenzie's picture

I received an email recently from a major funder of impact evaluations who wanted my advice on the following question regarding testing baseline balance in randomized experiments:

Should we continue to ask our grantees to do t-tests and f-tests to assess the differences in the variables in the balance tables during the baseline?  

Weekly links June 23: VoxDev launches, uncountable Nigerians, a challenge to prospect theory, and more…

David McKenzie's picture

Weekly links June 16: why women earn less, teaching fintech, now class size doesn’t matter, and more…

David McKenzie's picture
  • Women in Economics at Berkeley has a great summer reading list of recent papers which look at the gender earnings gap in different ways, including short summaries of some very recently published papers in the AER, QJE, and JPE on this issue.
  • The NYTimes on how business schools are trying to teach fintech, although with no agreement on what this means or how to do it.

Weekly links June 9: the dangers of out-dangering, debating how to provide health care for the poor, fighting corruption, and more…

David McKenzie's picture
  • Milli Lake and Sarah Parkinson on the ethics of fieldwork preparedness – “It’s one of the discipline’s worst kept secrets that graduate students, in particular, feel practically unprepared for their fieldwork… We worry about an intellectual trend that increasingly rewards researchers for “out-dangering” one another (often with dubious scholarly gain). This doesn’t mean scholars should abandon fieldwork; it means that we should take the practical and ethical components of its planning and implementation more seriously. We can start by asking simple questions about first aid, check-ins, transport safety, and data protection”

Weekly links June 2: do you need to correct your p-values for all the tests you run in your life?, nimble RCTs, the elusive entrepreneur, and more…

David McKenzie's picture

Weekly links May 26: the Chetty production function, collect priors before you work, small samples bring trouble, and more…

David McKenzie's picture

Weekly links May 19: another list experiment, P&P highlights, government nudges, and more…

David McKenzie's picture
  • The papers and proceedings issue of the AER has several papers of interest to development economists, including:
    • Esther Duflo’s lecture of “The Economist as Plumber” – “details that we as economists might consider relatively uninteresting are in fact extraordinarily important in determining the final impact of a policy or a regulation, while some of the theoretical issues we worry about most may not be that relevant”…” an economist who cares about the details of policy implementation will need to pay attention to many details and complications, some of which may appear to be far below their pay grade (e.g., the font size on posters) or far beyond their competence level (e.g., the intricacy of government budgeting in a federal system).”
    • Sandip Sukhtankar has a paper on replications in development economics, part of two sessions on replication in economics.
    • Shimeles et al. on tax auditing and tax compliance experiments in Ethiopia: “Businesses subject to threats increased their profit tax payable by 38 percent, while those that received a persuasion letter increased by 32 percent, compared to the control group.”
    • 4 papers on maternal and child health in developing countries (Uganda, Kenya, India, Zambia).
  • Following up on Berk’s post on list experiments, 538 provides another example, using list experiments to identify how many Americans are atheists.
  • The Economist on how governments are using nudges – with both developed and developing country examples.
  • The equivalent to an EGOT for economists? Dave and Markus have come up with the EJAQ or REJAQ for economists who have published in all the top-4 or top-5 journals.
  • Call for papers: TCD/LSE/CEPR conference on Development economics to be held at Trinity College, Dublin on September 18-19. Imran Rasul and I are keynote speakers.

Pages