· On the Data Blog, Alaka Holla, Laura Becerra Luna and Meyhar Mohammad ask are researchers good at sharing data even when they do make it available? Based on the experience of projects funded through SIEF, they note lots of common issues around not labelling variables, not including questionnaires, inadvertent inclusion of identifying information, and more… and provide some resources for those looking to improve the way they share.
· In the FT, “how flawed data is leaving the UK in the dark”, discusses the problems arising from a large decline in the response rate to the labor force survey (LFS) “Declining response rates to the LFS have made the numbers so volatile that it is impossible to be sure whether employment is rising or falling from one quarter to the next — let alone how the labour market has evolved in the years since the pandemic.” The response rate has fallen from 52% in 2014 to only 14.8% in 2023, which as well as being a massive drop, is much of a drop than in other European countries and the U.S. The article notes that poor data already seems to have led to policy focusing resources on the wrong problem – it appeared that there was a surge in people not working because of poor health, but it actually seems that just more people were claiming benefits without reducing activity. It also notes that part of the problem appears to have been an attempt to pivot from telephone surveys to online and administrative data, which not delivered as much as intended.
· A new working paper by Briggs et al. uses LLMs to examine over 2,600 political science articles published in two leading polisci journals over the 2010-24 period to examine trends in how research has evolved. Preliminary results find the best LLMs are about as good as the best graduate and undergraduate coders (at 1/100th of the cost, and 100x the speed). Among the results that may be of most interest to our readers: (i) the strong geographic concentration: “Of the 2,674 articles in our sample, 1,521use data from the United States”. UK, Germany, France, Denmark round out the top 5. Mexico, Brazil, India, Colombia and Argentina are the most studied developing countries; (ii) these political scientists love their survey experiments “In both journals, survey experiments have outpaced other ex-perimental approaches, reaching approximately 15–20% of all articles by 2023”; (iii) pre-registration rose from near 0 in 2015 to around 30% of published papers in 2023; (iv) “selection on significance remains high. 98.8% of abstracts explicitly report a non-null result. In contrast, only 16.9% of articles report a null result in the abstract. In very few (1.1%) articles do the authors report only a null result in the abstract.”
· The latest issue of the JEP has this excellent practical guide to shift-share instruments by Borusyak, Hull and Jaravel. It comes with a very nice summary Table 2 and a checklist of when and how to use (and when not to use).
· Call for Applications for a consultant to conduct a Systematic Literature Review on Nimble Evaluations (incl. A/B Testing) for Better Program Implementation: The European Regional Competitiveness Policy Lab (EuroCOMP Lab), a partnership between the European Commission and the World Bank, is seeking researchers to conduct a systematic review of nimble evaluations with relevant insights for better program implementation and policy making in the private sector. This study has the goal of identifying innovative and successful approaches for policymaking as well as summarizing evidence gaps for more future research. Experienced researchers in systematic reviews, firm behavior, and impact evaluations are invited to apply by submitting their CV, cover letter, and relevant work samples. Learn more HERE.
Join the Conversation