As a PhD student in the late 90s, randomized field trials were not yet common place in empirical development economics. Certain quasi-experimental methods such as regression discontinuity were also fairly exotic. It was the era of the “natural experiment”, when fellow PhD students scoured county newspapers at the university library for research leads. These students were looking for news of policy changes that might plausibly introduce some exogenous variation in the local market environment.
A difference-in-difference that contrasted changes in the outcomes in these “naturally treated” areas with neighboring locales yielded the estimate of impact. At seminars, much discussion focused on the plausible exogeneity of the natural experiment and questioners often went off on a search for equally plausible yet unobserved confounders. We don’t get many natural experiments at our seminar series anymore.
I was reminded of this legacy as I reread this paper that explores the relationship between poverty and mental health in children by E. Jane Costello and co-authors. It was published 8 years ago in a leading medical journal but flew under the radar in the economics community presumably because it was written by epidemiologists for the medical and public health community.
Also the study focused on the relationship between poverty and mental health - not a common cross-over area of interest in our field. However it is a long standing interest of mine. And it’s a nice example of what can be learned when researchers get lucky with an unanticipated change in the environment under study.
In the middle of an 8-year study of mental illness in children in the Smoky Mountains region of North Carolina, a casino opened on a Native American reservation that fell in the study area. The casino paid a percentage of profits to all tribal households. The casino and surrounding motels and restaurants also became a source of employment. Roughly a quarter of all children in the study was Native American and resided on the federal reservation, so there was sufficient density in the data to contrast changes in the Native American population with the neighboring white population that didn’t receive these direct transfers.
Children living in poverty are more likely than non-poor children to have a psychiatric disorder. In the baseline study data, children below the poverty line were 59% more likely to have a psychiatric symptom than non-poor children. However the problem of disentangling the relational direction of poverty and mental health is clear. It’s possible that the adversity and stress of poverty can lead to worse mental health, but it’s also possible that causation can run in the other direction – poor mental health of adults can lead to adverse economic outcomes and may also be transmittable to children.
Enter the casino and the annual transfers of up to $6000 per year to each reservation household. Poverty rates declined significantly. In these same households certain dimensions of child mental health, notably conduct disorders, improved significantly over a short period. (Although, importantly, other dimensions of mental health such as depression did not improve). The one significant mediator of the observed change in child health status appears to be an increase in parental supervision and parental presence in the child’s life.
One aspect the study can’t address is the causal mechanism behind this observed change in child outcomes and presumed quality of parenting – is the improvement a result of the increased income itself, or perhaps a result of the adult mental health benefits from increased employment? But then again, many experimental and quasi-experimental studies face this same interpretive difficulty.
Since natural experiments are almost never truly random, there is always the possibility of mis-attributing a relation as causal when in fact an unobserved confounder is truly driving the relation. In this case it’s unlikely that the observed changes are a direct result of the presence of a casino. For example, I doubt that adults make more attentive parents once they are happier because they have the option to gamble locally.
There is other work in public health that leverages the appearance of casinos to explore the relationship between income and health, however I don’t know of any work in economics that uses this particular type of “natural experiment” to explore, say, changes in consumption patterns or changes in risk preference, as a result of increased and sustained income transfers. I wouldn’t be surprised if the work is out there, I just haven’t found it.
Of course many of these same questions are now explored through impact evaluations where we either control the assignation of treatment directly or have a very good idea of the factors that determine treatment assignation. But let’s keep an eye out for the serendipitous casinos that may appear out of the blue.
Join the Conversation