It used to be that economics research papers on China were more common than those on India: Das et al. report that over the 20-years between 1985 and 2004, the top-5 economics journals published 39 papers on India, versus 65 on China. But my impression was that this pattern has greatly reversed with the rise of more fieldwork and impact evaluations, especially for graduate students. To investigate this, I used the AEA RCT registry (thanks to Keesler Welch for help on this) to see how many trials had been registered for each country: in total, there are 42 for China, compared to 175 for India and 120 for Kenya. The figure below shows the number by year, showing that China’s numbers grew much less in the mid-2010s than these other two countries.
Number of RCTs registered in AEA Registry by start year and country
It used to be that much of the work I did see on China seemed very China-specific – looking at the impacts of one-child policy, or of the Hukou system for migration, or some facet of the cultural revolution. But recently, I’ve been reading several interesting impact evaluations in China that address more general economic issues. Examples include:
· Jing Cai and Adam Szeidl’s QJE paper on interfirm relationships, in which they randomized firms to form business networks; and their newer working paper on direct and impact effects of finance on SMEs, in which they are able to measure spillover effects at the market level of competitor firms getting loans.
· Nick Bloom, James Liang, John Roberts and Zhichun Ying’s QJE paper on working from home, in which a Chinese travel agency randomized whether their workers could work from home or not.
· A new experiment I recently saw presented by Jie Bai, Maggie Chen, Jin Liu and Daniel Xu looking at t-shirt sellers on a global e-commerce platform, and the role of initial demand in generating future sales (paper not yet available)
· Victor Couture, Ben Faber, Yizhen Gu and Lizhi Liu’s forthcoming AER Insights paper on the impact of expanding e-commerce to rural households
This is on top of a large program of research by Scott Rozelle and the REAP team, which have been working on health and education interventions in rural China. In addition to RCTs, there also seem to be more non-experimental impact evaluations emerging, such as this paper on corporate income taxes to stimulate R&D investment, and this recent AEJ Economic Policy paper on congestion, traffic density and road pricing after a driving restriction policy. Given all this, I got to wondering how doing impact evaluations in China compares to other countries, and how easy it would be for someone to start working there. I realize that now may not be the most obvious time to be thinking about working in China with the Corona virus shutting down travel, but this post is intended for longer term thinking about research projects.
Since I have no experience working in China, I reached out to several experts who have. Thanks to Jie Bai, Jing Cai, Scott Rozelle and two anonymous researchers for their helpful comments.
Why might you want to work in China?
Jie: I would say it’s the “rich heterogeneity” within the country (or even within a province), easily spanning a wide spectrum of development stages, from ultra-poor villages to world-class high-tech manufacturing hubs. This potentially allows researchers to test for a richer set of economic theories and get at the external validity question (i.e. how does the treatment interact with other socioeconomic factors). Relatedly, many industries are undergoing fast growth and quality upgrading, providing a fertile ground to test many existing models of firm dynamics as well as inspire new theories.
Jing: China is big, has a large number of administrative units and population with great diversity. So it's easy to find enough sample to ensure power, and to get heterogeneous sample to improve external validity. Moreover, in recent years government officials have been increasingly open to collaboration with researchers and taking policy suggestions from them.
Scott: China is a unique and totally viable platform for studying the problems and dimensions of economic development (many unique or on the cutting edge), writ large: healthcare delivery, inputs to education, urbanization, poverty alleviation, early childhood development, firms and productivity, returns to education and health, etc. In several of these areas where studies depend on the presence of tech or other infrastructure, it’s the only place in the world to do it. On top of that, it is a place where the success or failure to address these problems (see: corona virus) will reverberate around the world.
One of the anonymous researchers I spoke to raved about the incredible data being collected – for example, minute-by-minute productivity data in large factories that enables studying questions that might be difficult to measure and study elsewhere.
Great, so can I just follow the common grad student model of finding an NGO who wants to work with me, or hanging out for a couple of months in the summer and talking with people and starting some interviews?
No. First of all, language can be a big barrier to entry. Second, in general, NGOs have a hard time operating in China and the laws regulating NGOs are quite restrictive. The few NGOs that do operate are under a lot of political scrutiny, and can be worried about attracting unwanted attention.
All the researchers I spoke with stressed the importance of local connections in order to conduct research. Scott noted that one key challenge is that China is very decentralized, so that it is not “simply” a matter of getting the permission of a central ministry or high-level government. For example, to do work in urban schools, even with ministry permission, each school principal would have to decide to opt-in. Jie notes the need for strong local political support for carrying out almost any type of household and firm surveys. For example, it’s very difficult (or nearly impossible in her view) for an independent research team to just go into a village to conduct household surveys without the approval of the village head, who would then very likely need an approval from the upper county government, and so on. Same for firm surveys. The political atmosphere may be worsening over the years — no one wants to "get into unnecessary trouble”, and there has been growing suspicion toward foreign research organizations recently.
Finally, one researcher noted there have been cases in the past of outside academics getting arrested for just going in and starting research, especially if working on topics deemed “sensitive”. So be careful.
Are there good survey organizations for hire, who can just take care of all this for you?
In some countries you can work with professional survey companies that know how to navigate these layers of permission, and can enable outside researchers to conduct surveys without large start-up costs. Several of the researchers noted the lack of organizations like IPA and J-PAL operating in China, and raised questions about the quality of data from private survey firms (“I heard that many survey firms fake the sample so I don't trust them”). The main survey efforts seem to be set up by academics associated with several Chinese universities. Examples include the China Household Finance Survey, and the China Health and Retirement Longitudinal Survey (CHARLS). Jing and Jie noted that their approach has been to hire and supervise college students to do the surveying – which requires speaking the language.
The risks of data collection getting interrupted by political concerns are seen in Couture et al’s paper, where they note “in one of the counties, the local government suspended our team’s data collection mid-way, leaving 4 of the villages without endline data”.
So what approach can work?
All of the researchers stressed the importance of local connections for being able to do work in China, and that this typically involves outside researchers seeking out local collaborators from Chinese universities who sometimes have these connections. Four different modes were suggested in my discussions:
1) Work closely with a large Chinese company or online platform: this has the advantage of potentially sidestepping the need for a lot of data collection, since you can potentially use large amounts of administrative data. Clearly this works better for some research questions than others – Scott noted that few companies are interested in/collecting data on the rural poor. But working with a large company or online platform may reduce the need to get lots of local government permissions. A couple of issues to consider here: i) companies may be reluctant to have negative results publicized, so being very careful what sort of MOU is signed with the company setting out permission to publish results regardless of how they turn out is important; and ii) be aware that the objective functions of companies may be very different from yours – one researcher told me that after working on an RCT in a large Chinese factory for several weeks, the company realized it was saving lots of money through the experimentation, and so hired a full-time team of experimenters and dismissed the researchers “You think you are doing an RCT, but they are doing a bandit” was the nice quote on this. But this still seems the lowest barrier to entry.
2) Work with an academic from a Chinese university: Jing notes “Local researchers have great incentive to collaborate with foreign researchers to improve their publication records. It will make things easier by partnering with local researchers and universities, because they can offer a lot of help on recruiting enumerators, contacting the government, and survey implementation. I always work with a local university to hire their students to work as my enumerators.” There are an increasing number of Chinese academics trained in top institutions in the US and Europe, who face increasingly difficult tenure requirements that require them to be publishing well, given them some increased incentives to collaborate on interesting research. One thing several researchers noted is that papers may only count for their tenure cases if these authors are listed as the first author. Jing notes that there are a number of workshops organized by Chinese universities in the summer, along with some visiting programs for foreign researchers, and both of these can be good ways to form contacts and brainstorm with local researchers.
3) Form a long-term collaboration: this is the model Scott has been pursuing for the past 30 years, in which he has built up a network of collaborators, including former students and students of students over time, and a team of experienced surveyors etc. who can help in running projects.
4) Collaborate with those doing work in China already: rather than start from scratch on 1)-3), you could learn from those who have already overcome some of the challenges.
I am excluding here the case where you have your own connection (e.g. James Liang, one of the authors of the working from home paper, founded the travel agency that this study was run on) – if you have these connections already, this blog post is not needed.
How easy is it to get funding to do research in China?
All the researchers noted that it has been very hard, with China being considered too rich for a lot of development donors and foundations to want to fund research, although some have had success with getting smaller pots of money by outlining the lessons for other developing countries from learning from China. But the main two sources of funding appear to be i) Chinese research funding, which has been growing. However, this is not available to researchers outside of China, and even within China, may be more directed towards certain provinces/urban areas than poorer remote regions. If you are collaborating with Chinese co-authors, this is one source; ii) private donors: altruistic rich Chinese ex-pats have funded some of these projects. Of course, if you are relying mainly on administrative data, then the funding needs are much less, and you may be able to get by ok with just relatively little funding.
What is your comparative advantage?
One interesting point made by one researcher was that China was a bit more open to outside researchers coming in to do research in the early 1990s, when the quality and quantity of Chinese university researchers was much lower. But now there are so many well-trained Chinese economists, that it is not always clear what an outsider coming in has to offer that can’t be done by one of these local researchers. So thinking about what your comparative advantage is will help in thinking about what types of collaborations to seek out.
Readers: please let us know any advice I’ve missed here, or share your own successful or failed experiences with starting work in China.
Join the Conversation