· Results are out for a 3-year RCT that MDRC ran to evaluate a microfinance product in the USA with Grameen America, and the headline results look a lot more promising than most studies in the developing world. The RCT is with 1,492 women in 300 loan groups in New Jersey. “monthly net business earnings increased an average of $127 for members as compared to their control group counterparts” (relative to a control mean of $356), with significant increases in savings, reductions in material hardship, increases in creditworthiness, and in life satisfaction. The loans are $500-1500 loans, with an interest rate of 15-18%. Most borrowers were Hispanic or Latina, and loans were meant to be for business purposes. However, as Kelsey Piper notes, much of the increase in business ownership comes from treated women using direct selling and multi-level marketing, which raises a number of concerns about sustainability and impacts on their social networks. The full study notes that the impact is positive, but not as definitive (p=0.101) on one of the two primary outcomes, total net income from all sources. The report is very thorough and I haven’t had a chance to read completely, but here are a couple of things I wanted to know:
o Two reasons for the lack of impact of many developing country RCTs have been the loans haven’t necessarily gone to people running businesses, and that the first-stage impact on credit has been weak – either because lots of alternatives to the microloans were available, or because lots of those offered loans did not take them up. Here the sample is pretty much all small business owners, and the impact on getting a loan from any lender is 63 percentage points – i.e. this really provided new credit they would not get from elsewhere. Moreover, over three years they could get up to 6 loans as they repaid and took new loans.
o I was curious what survey response rates would be in a U.S. context. 71% answered the 36-month survey, and this is balanced between treatment and control. But there is just over 20 percent item non-response on total household income.
· Sharique Hasan, Rembrand Koning, Hyunjin Kim and the Innovation Growth Lab have a new quarterly newsletter called Experimental Notes that abstracts recent studies on entrepreneurship, innovation and strategy. Their second newsletter discusses in detail the trade-off between relevance and power when building samples for these studies: “ The first issue is a basic one that many of us face regarding how representative the sample is of the population we are interested in studying versus testing on a less representative but larger sample that increases our ability to estimate a treatment effect. Often, researchers start by generating an idea that applies to a specific set of people or firms. For instance, a researcher might be interested in how incentives affect the behavior of managers or how improved judgment might affect the performance of startups. But because of the difficulty of getting enough real startups in their study, they do the study with ‘aspiring entrepreneurs’ or their students. Sometimes this is the right move because we think the underlying mechanism will not vary across different populations. However, there are many instances where we think there will be substantial variation in mechanisms and so treatment effects across different populations. In these cases, choosing the larger but less relevant sample can sacrifice the main value of running a field experiment – to understand the real effects of treatment in practice in an actual population of interest. For instance, showing that peer advice affects an undergraduate in a hackathon with no business experience is not especially informative about the impact of peer advice for a founder with 20 years of experience and 30 employees, who has raised millions of dollars from an experienced VC. In this case, the researcher has to ask whether a study with 100 real entrepreneurs provides more insight about a mechanism, albeit with more statistical uncertainty than a study with 1,000 undergraduates.”
· Scott Cunningham interviews John List on YouTube: he talks about his background getting into economics, and his new book, and then from minute 43, about economists in tech firms – and John List breaks the news he is going to now become the chief economist at Walmart.
· Asjad Naqvi has an updated Stata-to-LaTeX guide on Medium, walking through how to export Stata output into customized tables in LaTeX, with an Overleaf document that can be used as a template.
· CReAM has a new Ukrainian Migration Information Hub – with background data on the demographics of the Ukrainian population in Europe and Ukraine, updated information on prevailing visa/asylum policies across Europe, and summaries of research evidence on policies to aid refugee integration.
Join the Conversation