Syndicate content

Recent comments

  • Reply to: When bad people do good surveys   2 days 18 hours ago
    Great comments Jeff, thanks.    Indeed, checking for anomolous distributions is key -- one example that was in the Finn and Ranchhod paper was of one enumerator who wasn't using the anthropometric equipment right -- that wasn't fabrication, it was a case for retraining.    Getting this early is really helpful.   

    I also wholeheartedly second your point on optimal probing -- it's a tough spot to find -- if you push too much not only do respondents make things up, they sometimes get really annoyed.   We had this with an information question we were asking -- where we really wanted the team to probe.   It turns out there was an underlying norm about not explicitly discussing certain things (between individuals in the community, not with the survey team) and that to admit to discussing this was a no-no. 
  • Reply to: When bad people do good surveys   6 days 5 hours ago

    Great post, and one that most grad students starting out in development research could really benefit from. Something that this post brings to mind, and that researchers might do well to note, is that it can be really beneficial to check the distribution of responses to questions by surveyor for all questions, not just the ones that activate skip patterns or might be prone to fraud. There are some questions where a certain amount of probing is required to get a proper answer, such as "how many loans have you taken in the last month?" or "If you needed a quick loan of $100 who could you ask?". While we try to avoid that as much as possible in survey design, it is usually unavoidable. Surveyors may exert different amounts of effort in getting a response, ranging from too little (no loans) to too much (respondent feels pressured to make up loans). Probing too much or too little isn't necessarily a sign of fraud, but something that you want to standardize across surveyors; looking at your data after a week can help you figure who which surveyors you need to retrain in doing more or less. A second benefit is that surveyors sometimes misunderstand subtle distinctions in questions without deliberately meaning to commit fraud. For example, I had a survey where we were asked women if they had received antenatal care. When we looked at our data, we realized that one surveyor was putting "yes" much more often than others because she had misunderstood what antenatal care entailed. Our supervisors/backcheckers hadn't yet picked this up, since the misunderstanding only was apparent some of the time. Finally, this can help in giving specific feedback to surveyors in a way that both lets them know that you are checking on them carefully, helping to lower incidence of fraud, but is more constructive than conversations revolving around backchecks, which in my experience often create a lot of tension.

    In general, I think that this is one the main benefits of electronic surveying: getting the data back instantly allows you to find patterns that otherwise could be missed in scrutiny and even backchecking. With backchecking, the sample size is usually too small to detect these subtle errors, especially since backcheck forms often omit the questions where we know the responses are likely to be unstable. And researchers are looking into ways to improve fraud detection within survey software in some pretty interesting ways, e.g.

  • Reply to: Do the Poor Waste Transfers on Booze and Cigarettes? No   1 week 20 hours ago

    I was involved in a scoping study for a cash transfer project in Tamil Nadu in 2006 in Tsunami affected areas. A number of donors were already there and the word on the street was that the number of alcohol shops in the area had quadrupled after cash transfer programmes started. Just anecdotal evidence but made us think.

  • Reply to: Using lab-in-the-field experiments to predict and understand new product take-up: evidence from helping Filipino migrants send remittances for education   1 week 21 hours ago

    Seruously, you don't forsee any ethical issues with this agenda? Emotional manipulation of at risk and vulnerable folks in developing countries? Should givr Facebook's Data Sciences folks a run for their money.

  • Reply to: Do the Poor Waste Transfers on Booze and Cigarettes? No   1 week 2 days ago

    Over the past 13 years I have been working on a variant of CCTs, savings groups where villages (most all of them women) save their own money, end it to each other over the year to for income soothing, stocking a business, buying animals, and divvy up their savings according to how much saved plus a share of the profits. I directed a program, called saving for change that trained groups with 650,000 members part of a larger savings group revolution with 10 million group members. No evidence of the money being misused but a major RCT showed decreased chronic hunger, increased assets, reaching the very poorest, the viral replication of the program into control villages by volunteers eager to share what they learned and increased social capital. The total investment per village of about 1,000 people, $1,000. This is a much lower cost intervention than CCTs. My book In Their Own Hands: How Savings Groups are Revolutionizing Development on savings groups and their impact will be published by Berrett Koehler in the fall.