Syndicate content

Development Impact turns one – tell us what year two should look like

Jed Friedman's picture

 

As we celebrate our first year of the Development Impact blog, we thought it would be a good time to take stock and see what our readers would like in our second year. We’ve already done our survey work and RCT, so now its time for direct, self-selected, feedback from you, the reader.  We want to know:

·         What sorts of posts do you want to see more or less of?

·         Are there particular people you would like to hear from in a guest blog post if we can persuade them to write?

·         Any technical questions or issues you’d like us to give our thoughts on?

·         Any topics you think we aren’t covering enough?

Thanks for helping make our first year a successful one, and we look forward to another great year.

 

Comments

Submitted by Anonymous on
If you refer to an article you should provide an ungated link or a copy attached. And I have not seen anything that talks about the feedback loop or how all your endless studies actually are incorporated into better development programming. Where is that evidence? By the way, your security system requiring the typing of "2 words" is impossible to read. They are not words but groups of letters. And the sound does not work.

Thanks for the comments! We do try to provide ungated links where possible, and will pay even more attention to this matter. A very good suggestion on the incorporation of results into development policy - we hope to give this more attention. Sadly it doesn't happen as often as one might like... We are as troubled by the security system as you, but unfortunately we do not have the ability to switch to a more usable system. All World Bank blogs work on this platform - and we have voiced the same concerns before...

I think it might also be very interesting to talk about all those frustrating failed attempts to incorporate impact evaluation findings into policy! I think we can learn a lot from these experiences as well.

I rarely comment on these blogs because they're so good. It's probably the thing on my google reader that gets e-mailed around to colleagues more than anything else. So my over-riding advice is to not change too much. I like the mix of reporting on new papers (and one of the great things about this blog is that you catch lots of papers very early, either in WP form, in early drafts or just before publication), reporting on your own experiments and studies and the meta-evaluation stuff. What I would like to see more of is the slightly more critical stuff - looking at what might be wrong or might be poorly designed in some of the papers you cite - it would be performing a service to the authors as well as the reader. In terms of topics, I'm keen to see two things: more on the final outcomes of CTs (i.e. not enrolment, but achievement and labour market outcomes) and more stuff on labour demand (as opposed to supply).

Ranil, thanks very much for the kind comments. When discussing the blog among ourselves, we often focus on the very issue you raise - suffice to say we will post critical thoughts where we feel it is warranted (and perhaps do so more often in the coming year) but predominantly we hope to highlight work that we find good or interesting. Thanks also for the suggestions for topics... good choices.

Submitted by Anonymous on
would love to see more posts on reduced form vs. structural approaches. rather than picking sides, it's particularly useful to see how these approaches can be complementary, ie papers that combine experiments with structural models, or at least work out the structural assumptions which map into the reduced form estimates (and vice versa).

Submitted by Marie Koerner on
Dear colleagues, I am a development practitioner- free lance consultant. Some of the assignments I participate in or lead are evaluations of development projects, occasionally programs. The questions for ex-post evaluations/evaluations of completed projects always include impact. These evaluations take place typically shortly after the project completion. For some types of projects such as rural water supplies this is too early; at the best the likelihood of impact can be assessed. I would prefer to see impact evaluations of rural WASH and similar projects carried out say 2-5 years after completion. Answers for the remaining evaluation questions can be sought earlier. It that at all realistic? Thank you for considering this issue.

Marie, thanks very much for the comments. There is certainly a current dearth of medium or longer run evaluations, in part because it is a new approach to doing development econ, at least on the scale it is now done today. There are also, often, unique challenges to assessing longer run impact after the intervention has run its course. Nevertheless this is a critical concern and we will be on the lookout for good examples...

Submitted by Larrú on
I would like to read more on real and practical difficulties doing RCT. Researchers and evaluators could give us information about difficult case studies -technical and political- and how they could lead with them (or not). Fridays links are useful and they "relax" some hiper-technichal posts. Maybe, people coming from the four J-PALs (Boston, Chile, France, India) can write from time to time. Howard White would be a great special guest.

Larru, thanks for the wonderful suggestions. We do indeed plan to discuss the nuts and bolts of implementing RCTs and the various practical challenges. We are also always seeking "guest" posts from colleagues, hopefully from some of the people/places you list. In the coming months, expect to see more of this...

Submitted by Bertha on
Hi Jed, I agree that talking about the difficulties of implementing RCTs would be a great topic. I would like to see some estimate of the failure or succes rate of early selection of projects. For instance, out of the SIEF or DIME workshop's cases, how many have indeed ended in an IE? Does anyone have information on this in the Bank? I would enphasize as well the important distinction between effectiveness and efficacy studies, or proof of concept implementation vs. "real-world" or government-led implementation. it is very different to deal with small controlled IEs than to deal with government IEs. I would like to hear more on Bank's practitioners experience, I know SIEF is not limited to government programs, in which very often adjustments and changes along implementation are out of evaluators' control. Hope to see more sharing on this. In my previous work I managed evaluations for the government of my country. We tried to capture some of the difficulties and stories(especially political inference) in carrying out IEs in section 3 of this report: http://www.3ieimpact.org/admin/pdfs_papers/Working_Paper_14.pdf We recommend to always to have a plan B when attempting a randomized design. Anyway, the blog is very welcome, keep it up!

Submitted by priya alvarez on
Dear authors, your posts are excellent. I somehow store them & read when I have the chance. I am mostly interested on gender issues and although not always completely taken by interpretations, yours offer one of the few insightful accounts of evaluation results for women / disaggregated data or even impact on overall gender equality picture. I love the examples, the experimental design, metaevaluative stuff. You are truly inspiring in your openness and level of detail. It is difficult to find anything remotely similar coming from most multilateral or bilateral institutions where I work and messages are carefully crafted, monitored and...can I say..censored. It is refreshing to find your posts and high quality evaluations. thanks & thanks!