Reply to: Weekly links April 28: how many qual interviews are needed, enterprise-academic collaboration, work for me, and more…
Namey's advice is useful I guess for survey design. But let's be clear that coming up with a list of topics is not a particularly meaningful objective of qual methods for most of us who toil with these tools, 80,000 hours notwithstanding.
Reply to: Is My NGO Having a Positive Impact?
Am so impressed.
But my question is, What impact does the NGOs have on economic outcomes like inflation, growth, employment and income distribution.
Reply to: Should we pay kids to read?
Thanks, Erik. On your point (1), in the blog post I say "Why do we want children to read books, anyway? Adichie wants them to read to 'understand and question the world.' We also want children to read books to increase their reading comprehension and to learn to love reading for both enjoyment and improved opportunities in later life."
We know that kids with more exposure to books also have better reading comprehension (several studies are cited in Guryan et al.), but that is not causal. Indeed, this work suggests that -- at least in some cases -- it does (unless one believes that somehow the incentives are working through some channel other than getting the kids to read books), at least for the already motivated kids.
On (2), I'm not sure I see the ethical problem. Are you asking if it's ethical to do an RCT on this (i.e., give incentives to some kids and not to others), or if it's ethical to pay kids to read at all. In either case, I'd be interested to hear your argument.
On (3), I don't know, since the authors don't provide a CEA (although Fryer at least reports how much money was spent in payouts). But with the Guryan et al. study, all that money to get kids to read 1-2 extra books in a summer? I'd be surprised if a CEA were favorable.
Reply to: Should we pay kids to read?
Great post as usual. Without the benefit of reading the articles, I am not surprised. There is obviously the issue of the "quality" of reading, which I guess is resolved by the Fryer approach although I doubt something like that would be brought to scale.
Two deep and one shallow comment.
(1) The blog post says that the goal is to get them to read more books. Period. I think you need to have some research to show that reading more books contributes to reading skills. Does it? Not a facetious quesiton at all.
(2) Is this ethical? I suppose you could argue that ethics is the second question, since there is no need to worry about if the whole approach does not work in the first place. But I am not sure that we can get off that easily. I remember what Sandel said about Homo Econimicus. (http://tannerlectures.utah.edu/_documents/a-to-z/s/sandel00.pdf)
(3) Related to comment 1. Is this worth the money? With some sort of understanding of WHY we want them to read in the first place, how we can really value the program? Maybe some sort of CEA?
Reply to: The importance of study design (why did a CCT program have no effects on schooling or HIV?)
Thanks for the comment.
On the issue of you changing your mind, obviously that is not necessary: given theory and other extant evidence, the findings from one extra study may do very little to change our priors about a causal relationship - that's OK. What is not OK, however, is to have a sentence on the first page of the paper (under the Interpretation sub-section of the Summary) that goes: "Keeping girls in school is important to reduce their HIV-infection risk." This does not follow from your RCT. Put in other words, if this paper can make that statement in its abstract, so can any other paper that contains data on HIV and school enrollment status. Even if the authors put that sentence in the abstract, the editor should not have allowed it. I thought that biomedical journals were much better at sticking to a reporting template and minimizing speculation.
On hindsight being 20/20, sure, that is true. However, it's hard to chalk up the large number of things that accumulated to produce the null results as coincidental and all due to back luck - some of it, such as study setting and eligibility for the target population could have certainly been adapted between conception in 2008-09 and the actual start of recruitment in 2011 (about five years prior to eventual publication). Similarly, UCTs could be part of the trial...
Finally, yes, potential impacts on gender-based violence are important. But, the paper disinuguishes itself from the rest of the literature by its focus on HIV incidence, rather than self-reported sexual behavior or HIV prevalence. Then, the findings on the secondary outcomes must be subject to the same scrutiny on self-reported outcomes elsewhere in the literature.