Published on Development Impact

Weekly links March 12: LLMs for papers and lessons from using them for DiD, uncovering economic trends in places without much data, UBIs vs job guarantees, and more…

This page in:
Young boxers at the White Collars Boxing Match 2019, taken by Mariajose Silva Vargas

·       On VoxDev, Stephan Haggard, Kyoochul Kim & Munseob Lee discuss the range of forensic methods available to researchers for studying economies like North Korea, where official statistics are unavailable or unreliable: “researchers can use a range of forensic methods – such as satellite imagery, mirror trade data, price monitoring, refugee surveys, humanitarian data, and text mining – to extract credible economic information… No single method is enough on its own. The most robust findings come from triangulating across sources”

·       Planet Money covers Abel Brodeur and his replication games – and interesting hearing the interviews with participants, where some are at least publicly saying we just want to find whether the paper checks out, whereas others say their hope/goal is to find something wrong. There is also the hypothesis that “Several replication gamers told us their experience here will change how they do their research, because they know that their papers, too, might someday end up under Abel's spotlight.” – will be interesting to know what the treatment effect of participating in these replications is on both the replicators and on subsequent papers from those who have been through a replication.

·       Paul Goldsmith-Pinkham offers a proposal on how authors should augment their papers to format them in a way to make it easier for LLMs to read them. “First: an llms.txt file for papers—an author-curated orientation that tells an LLM how to read a paper before it starts reading. At its simplest, a few paragraphs: what the paper shows, what it does not show, where to look. Second: a lightweight paper bundle—a zip archive containing the paper in markdown alongside figures, data, and the llms.txt—so the full content is available in clean, token-efficient form”

·       Where do PhD economists land – website that summarizes placement data from top economics departments over time.

·       Some lessons for implementing Callaway and Sant’Anna Difference-in-Differences after using Claude Code to implement in Stata, R, and Python from Scott Cunningham.  He notes that doubly-robust estimators have a propensity score baked into them, and this can cause estimates to differ across packages depending on how covariates are handled and how they handle cases where the propensity score gets close to 0 and 1. A couple of practical recommendations fall out of this. The first is the importance of standardizing covariates – he gives an example of an interaction between population and year, which is a big number – which then when you do matrix inversion causes problems with how things are rounded – so using z-scores helps on this. But the second is being clear on what happens when common support fails – e.g. if some combination of covariates perfectly predicts treatment assignment for some units. But also a nice illustration of how Claude can be used to do tasks that you would never otherwise try.

·       Max Kasy and Lukas Lehner discuss the pros and cons of universal basic income versus job guarantee policies in a developed country context, along with results from experiments in Germany.  They argue that rather than pitting one against the other, “these policies should instead be seen as complements, serving different purposes and solving different problems, for different parts of the population.” See this review by Martin Ravallion for a discussion of these policies in developing countries.

·       In Science’s Expert Voice, Imran Rasul discusses some of the issues with moving from field experiments to policy interventions at scale.


David McKenzie

Lead Economist, Development Research Group, World Bank

Join the Conversation

The content of this field is kept private and will not be shown publicly
Remaining characters: 1000