A few of the many evaluation results on text messaging interventions
Text messages are being used for all kinds of things. SMS messages have been shown to increase adult literacy in the USA, improve pest management among potato farmers in Peru, increase adherence to malaria test results in Nigeria (don’t take anti-malarial drugs; you don't have malaria!), increase adherence to antimalarial drugs in Ghana, increase adherence to antiretroviral therapy for AIDS patients in Kenya (also in this other study), reduce drug stock-outs in Kenya, improve timely immunization in Kenya, improve glycemic control among diabetes patients in Bangladesh, improve school attendance in Brazil, and even increase risky sexual behavior and infidelity in Uganda (that one was an unintended consequence, not a bungled policy application of Steven Landsburg’s “more sex is safer sex” proposal, inspired by Michael Kremer’s old research).
These interventions are particularly interesting as mobile phone penetration continues to grow in low- and middle-income countries. In Ghana, the
number of active mobile phone lines passed the estimated population in 2012. A
survey last year from the Pew Research Center found that more than 80% of respondents had either a cell phone or a smartphone in Ghana, Kenya, Senegal, Nigeria, and South Africa, with Tanzania not far behind. The same survey found that young people are particularly likely to use cell phones to text, suggesting that SMS interventions might be an effective way to reach youth.
Along those lines, I just read an interesting study by Rokicki et al., “ Impact of an interactive mobile phone reproductive health program on knowledge and behavior among adolescent girls: a cluster-randomized trial.” Rokicki et al. test whether sending text messages to adolescent girls in Ghana can improve knowledge about reproductive health and – more importantly – change behavior. It looks like it can, and the positive impacts endure for a year after the intervention.
One element of the study points to increased potential for interesting SMS-based interventions: Whereas most of the examples above involve one-way text messages, Rokicki et al. incorporate an interactive model that proves twice as effective as the one-way model.
Here’s a brief recap of the study. About 750 adolescent girls (aged 14-23) across thirty-eight Ghanaian secondary schools were randomized – by school – into one of three groups:
Group 1 – Unidirectional intervention: Participating girls in these schools received one reproductive health fact per week by SMS over the course of 12 weeks. Below is one of the facts they received.
Group 2 – Interactive intervention: Girls in these schools received a true or false question once each week.
If they replied (free of charge), then they received a follow-up message.
If they didn’t reply, they received a reminder to reply on Wednesday and again on Friday. If they didn’t respond at all, they received a message at the end of the week with the correct answer. For every two correct answers, participants received a reward – an airtime credit worth 1 Ghanaian cedi (about 25¢ US).
Group 3 – Control. Don’t worry: They received messages too! Their messages were “placebos,” true facts but focused on malaria and unrelated to reproductive health.
Girls in both groups 1 and 2 received a tip that they could call a hotline with any health questions. All messages were automated.
Data. Participants took a test gauging knowledge of reproductive health at baseline, at the end of the 3-month intervention, and one year after that. They also answered questions about pregnancy and sexual behavior. Attrition was low (95.4% of baseline participants took the test 18 months later). Participants look balanced across groups.
Results! At the end of the intervention, the knowledge increase was 11 percentage points in the Unidirectional intervention and more than twice that – at 24 percentage points – in the Interactive intervention. One year later, the control group had caught up to the Unidirectional group (these girls aren’t living in a vacuum, after all), but the Interactive group maintained a lead. The authors highlight that this is exactly the period when many girls in Ghana initiate sexual activity ("by age 15 only 8% of girls have had sex, by age 18 about 43%, and by age 20, about 71% have had sex"), so a one-year head start in knowledge could be significant.
Knowledge scores from Rokicki et al. (2015)
In terms of behavior, participants in the Interactive group were slightly more likely to have had sex in the last year (21% in control versus 27%), but the probability of pregnancy – conditional on having had sex in the last year – was significantly lower for both the Interactive and the Unidirectional groups (8% for both, versus 16% in the control group). Adolescents in the Interactive group were more likely to have sex without a condom but also more likely to have used birth control. As the authors put it, “It is possible that girls felt emboldened by the information they received and thus had more sex, while also protecting themselves more effectively from unintended pregnancy.”
There are obvious limitations, not least that the data are all self-reported. Also, the Interactive arm is different from the Unidirectional arm in more than simply its interactivity: It includes a (small) incentive and – if participants don’t respond – more follow-ups, so we can’t sort out exactly why the Interactive arm is working. As usual, I’d like to see some cost analysis: How much more did the Interactive arm cost to run? (Since it was all automated, hopefully not much.)
But this kind of fully automated, interactive SMS intervention has implications not only for reproductive health education or general health education, but for other kinds of education as well. I look forward to seeing what it can do.
“D’oh! I got the true-false question on female condoms wrong!”
Photo from Wikimedia Commons, originally uploaded by Vox Efx.
Join the Conversation