Syndicate content

Recent comments

  • Reply to: What’s New in Measuring Subjective Expectations?   1 day 22 hours ago

    Thanks for the great post (not that I needed more things on my reading list...).

    On behavioral biases outside the lab, some co-authors and I are working on a non-lab paper looking at the impact of being struck by an extreme natural disaster on individuals’ risk attitudes, as well as their subjective expectations of future shocks. We find that being struck by a cyclone increases future expectations of losses, but that the whole sample on average has "too high" expectation of future losses from natural disasters, compared to the predictions of climate and hydrological models.

    One of the difficulties of looking at these biases outside the lab (at least in a context as noisy as natural disasters) is missing pieces: knowledge of the true probability distributions, and whether individuals consider these distributions as known or unknown are pieces that we wish we had. For this reason, we try to categorize the signs that different risk models would predict for the impact of a one-time shock on subjective expectations and risk aversion (the literature has found pretty disparate impacts, mostly on the latter, in the context of natural disasters).

    We speculate a bit about what might be going on by comparing the two ethnicities in our sample but, again, to really be able to separate these effects outside the lab requires a lot of info...

  • Reply to: Electronic versus paper-based data collection: reviewing the debate   4 days 15 hours ago

    Errors without bias

    Thanks for this post, guys! Great stuff. Just wanted to push gently back on one statement here about errors in the data: "As long as these are randomly distributed, we usually need not be overly concerned." This is a common sentiment that I think arises in part from our training in statistics, classical measurement error vs. bias, etc. -- but I've come to think that it might be a bit dangerous/misleading in practice. Samples are generally limited, statistical power is nearly always less than one would like, and in the end noisy zeros and other imprecise results have plagued nearly every impact evaluation I've seen up-close. The damage done by imprecision is, I think, massive. One strategy is just to invest in larger samples, but another is to do more to systematically reduce even mean-zero errors so that, for a given sample, results are more precise. I suspect that many investments in greater data quality can be much more cost-effective, per unit of standard-error reduction, than increases in sample size. Sorry -- I know that I'm preaching to the choir here, but I just wanted to add this comment to the discussion! Thanks again for sharing this!

  • Reply to: Electronic versus paper-based data collection: reviewing the debate   4 days 16 hours ago

    Thanks for the great blog post. A related post that readers will find useful is about measurement and issues around the quality of collected data on IPA's blog:

    And really interesting comments too. Sarah, your point about the risk-averseness and concerns about changing from PAPI to CAPI that teams on the ground have is well-taken.

    I work at Dobility, which created SurveyCTO, and we're always also trying to figure out how to address these constraints to change. It's definitely true that the nature of the work for field managers/supervisors ends up changing and I think a good approach is to invest a little bit in training them to adopt new roles: they may no longer need to be scrutinizing individual paper surveys for skip pattern errors but digital data collection software can also make it much easier for them to review aggregated data, generate reports of surveyor performance and errors (even without advanced statistical knowledge), and act on that data at a higher level to manage the team and improve its performance. So their work could actually shift to something that builds a broader skill set and is as or more productive, even if there is an initial learning curve.

  • Reply to: Electronic versus paper-based data collection: reviewing the debate   4 days 18 hours ago

    Thank you Anne for these additions. Good point on logistics. And thank you for the idea for a follow-up post.

  • Reply to: Electronic versus paper-based data collection: reviewing the debate   4 days 18 hours ago

    Thank you Segun for sharing this perspective from a data firm.