Violence against children remains one of the most widespread yet often normalized forms of violence globally. Parenting programs that promote positive discipline have shown promising results in reducing harsh discipline and improving child development outcomes. However, most evidence-based programs rely on in-person delivery, which can be costly and difficult to scale. As digital technologies become increasingly accessible, many researchers and policymakers are exploring whether these programs can be delivered digitally without losing effectiveness.
In a recent paper, “Violent discipline and parental behavior: Short- and medium-term effects of digital parenting support to caregivers,” just accepted at the Journal of Public Economics, my coauthors and I evaluate whether digital technologies can make evidence-based parenting programs more scalable while still maintaining their effectiveness.
To this end, we adapted an in-person Jamaican parenting program, the Irie Homes Toolbox, for digital delivery. Originally developed to prevent violence against children through behavior change techniques, the program targeted caregivers of children aged 2 to 6.
The version of the program we evaluated included three digital components. First, caregivers received three SMS messages per week containing simple parenting tips and reminders about positive discipline practices. Second, participants were given access to a data-free mobile application that included short videos and additional content reinforcing the parenting techniques introduced in the messages. Third, caregivers were invited to participate in weekly online group sessions led by trained facilitators from Jamaica’s Early Childhood Commission. To evaluate the program, we conducted a randomized controlled trial involving more than 1,100 caregivers of children aged 2–6 across Jamaica.
The results are encouraging. Caregivers who participated in the digital parenting program showed significant improvements in their attitudes toward violence against children and reduced their use of violent disciplinary practices. These behavioral changes were not short-lived: many of the improvements persisted nine months after the program ended. The intervention also improved caregivers’ parenting knowledge and confidence, suggesting that the program helped parents feel more capable of managing difficult situations with their children.
But rather than focusing on the results of the intervention implemented in 2021, I want to reflect on three challenges we encountered during the study. These challenges highlight some of the persistent gaps between the promise of digital interventions and the realities of implementing them at scale. They also point to areas where, five years later, new technologies (including AI) may help address existing barriers, while raising new questions for researchers and policymakers.
Challenge 1: Technology is not adoption
One component of the intervention was a data-free mobile application designed to complement the content delivered through the other two components. The app contained videos demonstrating positive parenting techniques and additional material that caregivers could access at any time.
From a design perspective, the app seemed ideal. It addressed many of the barriers that often limit digital interventions in low-resource settings. Because the app was data-free, caregivers could access it without consuming mobile data, which is a critical feature in environments where connectivity costs remain high. The content was concise and visually engaging. Parents could revisit the materials whenever they wanted, allowing them to reinforce lessons from the SMS messages and online sessions.
Most importantly, once developed, the app had essentially zero marginal cost per user. If widely adopted, it could dramatically increase the scalability of parenting programs.
Yet the reality looked very different: only 10 percent of participants opened the app at least once, and among those who did, average usage across the entire 12-week program was only seven minutes. These numbers were striking, especially given that participants had already expressed interest in the program by enrolling in the study. Why would caregivers who were motivated enough to sign up for the intervention rarely use a tool designed to help them?
To better understand this puzzle, we conducted focus group discussions with caregivers after the program ended. The conversations revealed several insights about how participants perceived and interacted with the digital tools we provided.
First, caregivers highlighted the small but meaningful frictions associated with logging into the app. Remembering passwords, recovering login credentials, and navigating unfamiliar interfaces may seem like minor inconveniences, but in practice they created barriers for busy parents managing work, childcare, and household responsibilities.
Second, and perhaps more importantly, caregivers did not view the different delivery channels as complementary. Instead, they saw them as substitutes. Because they were already receiving parenting tips through SMS messages, many participants felt that opening the app was unnecessary. The SMS messages were simple, immediate, and required no additional effort. From the caregiver’s perspective, they already had access to the core information.
This experience highlights a broader lesson: digital interventions often underestimate the importance of user behavior and preferences. Designing an app that is technically accessible does not guarantee that people will actually use it.
The challenge of low adoption is not unique to parenting programs. Many EdTech interventions have faced similar issues over the past decade. Despite rapid advances in digital tools, user engagement remains a persistent constraint.
This challenge becomes even more relevant as the development community increasingly turns toward AI-enabled solutions. From AI tutors to mental health chatbots, many new interventions promise to deliver personalized services at scale. But these technologies can only have an impact if users adopt and engage with them.
Recent evaluation frameworks for AI interventions, such as this one developed by the Agency Fund, emphasize the importance of testing not only whether a technology works in principle, but also whether people actually use it in practice. Evaluations that focus on adoption and engagement (levels 2 and 3) are therefore critical before deploying technologies at scale.
Our experience with the parenting app suggests that understanding how users interact with technology in their daily lives may be just as important as the technology itself.
Challenge 2: Measuring violence against children
A second challenge concerns the measurement of violence against children, which is inherently sensitive and difficult to observe. In our study, the primary outcomes rely on self-reported parenting behaviors. This approach raises an obvious concern: parents who know they are participating in a study about parenting practices may adjust their responses to appear more socially acceptable.
This concern was raised repeatedly during the peer review process for our paper. If caregivers in the treatment group felt that researchers expected them to reduce violent discipline, they might report improvements even if their behavior had not changed.
To address this concern, we conducted several robustness checks to examine whether experimenter demand or social desirability bias could explain the results. One approach involved constructing an index capturing each participant’s propensity to provide socially desirable responses. We then examined whether treatment effects differed between caregivers with high and low propensities to provide socially desirable responses. If social desirability bias were driving the results, we would expect stronger effects among participants with higher tendencies to provide socially acceptable answers. We found no evidence of such patterns.
Nevertheless, measuring violence against children remains an ongoing challenge. Alternatives are limited. Asking young children directly about their experiences is not feasible when they are 2 to 4 years old. External observation can introduce its own biases, as caregivers may alter their behavior when they know they are being observed.
Developing better measurement tools for sensitive outcomes like violence remains an important research frontier. New technologies may eventually help address some of these challenges. For example, with some coauthors we are exploring the potential of using computer vision tools to detect signs of emotional distress or victimization among adolescents through facial expressions or behavioral patterns. Such technologies remain in their early stages and require extensive piloting and validation. But they illustrate how advances in AI and digital sensing could complement traditional survey methods in the future.
Challenge 3: Collecting biological measures of stress
A third challenge emerged when we attempted to measure caregiver stress (a secondary outcome) using salivary cortisol, a biological marker commonly used in research on stress and mental health.
Biomarkers can provide valuable objective evidence to complement self-reported measures of well-being. In our study, we hoped that cortisol levels could help validate survey-based measures of caregiver stress and mental distress.
However, incorporating biological measures into field experiments can be extremely challenging.
The first barrier is cost. Processing cortisol samples requires laboratory analysis and specialized equipment. We were able to pursue this component of the study thanks to support from the World Bank’s Strategic Impact Evaluation Fund (SIEF). But even with funding secured, implementation proved difficult. We offered monetary compensation to participants for their time and transportation costs, partnered with a laboratory network across Jamaica to facilitate sample collection, and deployed mobile labs to make participation more convenient. Despite these efforts, we ultimately collected samples from less than 15 percent of the participants selected for the exercise.
One of the main constraints was the strict timing requirements for cortisol collection. To ensure comparability across participants, samples must be collected early in the morning, typically around 7 a.m. For many caregivers (and adults more generally) this coincided with the busiest part of the day, when they were commuting to work or preparing their children for school. In addition, providing biological samples can feel invasive. Even when participants trust researchers, the idea of collecting biological material can raise concerns or discomfort. These challenges illustrate the logistical hurdles associated with incorporating biomarkers into large-scale impact evaluations.
Once again, emerging technologies may offer potential alternatives. For example, with some coauthors we are exploring whether voice biomarkers could help diagnose mental health conditions such as stress, anxiety, or depression. By analyzing vocal patterns through short voice recordings, these tools could potentially provide low-cost diagnostic signals, without requiring biological samples, that can be used reliably in policy contexts.
Looking ahead
Digital tools are rapidly transforming how development programs are delivered and evaluated. Parenting programs, mental health interventions, and education initiatives increasingly rely on mobile phones, apps, and emerging AI technologies to reach large populations at low cost. But our experience implementing this digital parenting intervention highlights an important lesson: technology alone does not solve implementation challenges.
Ensuring that users adopt digital tools, improving the measurement of sensitive outcomes, and developing scalable ways to capture well-being remain key priorities for future research. As enthusiasm around AI continues to grow, rigorous experimentation and careful evaluation will be essential to ensure that technological innovation translates into meaningful improvements in people’s lives.
Join the Conversation