Today’s installment in this
occasional series on how to use your development economics PhD outside of a research university is with
Evan Borkum, a senior researcher in the International Research Division of Mathematica Policy Research Inc.
DI: Please provide a short paragraph describing what you do in this job, and give us a sense of what a typical day or week might look like for you. My job is to conduct independent rigorous impact and performance evaluations of social programs in developing countries. Most of this work is conducted under contract to US government agencies (mostly MCC and USAID) and various foundations, who issue requests for proposals to evaluate their programs. In my eight years at Mathematica I’ve worked on evaluations in Asia, Africa, and Eastern Europe, and in topic areas including agriculture, primary education, vocational training, maternal and child health, land, and others. As senior researcher on an evaluation team I’m typically responsible for technical leadership of all aspects of an evaluation, including study design, data collection, and final analysis and reporting. Last week was fairly typical and included work on designing a randomized controlled trial of an anti-child labor program, drafting a quantitative survey of vocational education students, and planning the analysis of survey data from farmers in Morocco.
How much of your time do you spend a) doing research; b) reading research done by others; c) travelling in developing countries? I spend the vast majority of my time doing research, working on a handful of different evaluations at any given time. In terms of reading research by others, we typically do a formal literature review at the start of an evaluation to inform the study design (for example, statistical power calculations) and eventually to contextualize the findings. I also regularly read research conducted by other Mathematica teams as part of our internal quality assurance process, which is essentially internal peer review. Travel-wise, I travel to developing countries about four times a year, typically for a week or two per trip. These trips are short but really valuable in getting a taste of the local context and program implementation, as well as building relationships with local stakeholders. Besides the activities you mention I also typically work on a handful of proposal efforts each year to help us bid for new contracts, which typically last a few weeks each.
How easy is it for you to find time to take projects from the “deliver to client” stage to getting them through to peer-reviewed journal publication? I imagine you need to charge your time to specific grants/projects, and that once the report has been delivered to the client, it may be hard to then charge time over the next year or two to take it to the publication stage. We are definitely encouraged to publish our findings in peer-reviewed journals, and there are some internal resources available to work on publications if we can’t use contract funds (in some cases we can). The challenge is often just carving out the time to do the work, as one tends to quickly get busy on new projects. So in practice we probably don’t publish quite as much as we would like to, though it is a goal that we are working towards.
A related question is whether everything you do ends up in the public domain, or whether some of your evaluations are delivered to the client but then never released publicly?
Virtually all of our final reports are published on our website. In addition to reports, we also produce public or restricted use data files for some projects, which the client makes available for use by other researchers.
Tell us about one accomplishment in this job that you are particularly proud of. Although Mathematica has been working on international evaluations for more than a decade (and domestic evaluations since the 1960’s), our international research division was only founded a few years ago. It brought together staff from across the company who were passionate about international development into a single structure where we could collaborate more closely and expand our international work. I’m proud of the contribution that I—and all my international division colleagues—have made to the success of our division since it was established, by producing consistently high quality and informative research for our clients and stretching ourselves into new areas. The volume and scope of our international work today is really unrecognizable from when I started back in 2010.
How did you end up in this job? What do you like most about it, and what are the main downsides compared to being in a research university? Mathematica was actually my number one choice coming out of my econ PhD. My adviser had consulted for Mathematica on one of their first big international projects and told me about them; it sounded like a great opportunity to do something more practical with my PhD and avoid several years of stress worrying about tenure. In terms of the work itself, there are several similarities with academic research. I think the quality of our research is just as high from a technical perspective (we really care about getting the standard errors right!), which is important to me. Like academics, I also get to work on studies that interest me—we tend only to bid for work in the first place if we can form a team of interested staff. However, an important advantage is that I don’t have to “sell” my research to anyone as being interesting, as academics have to do. I think that it’s interesting simply because our clients are investing hundreds of thousands or even millions of dollars on these programs and need to know whether they work. I also really like my colleagues on a personal level, which makes for a very collaborative and supportive work environment. The only downside I can really think of compared to academia is less flexibility in one’s schedule, though Mathematica does offer a fair amount of flexibility relative to other private sector companies (I actually work remotely from Bangkok these days for family reasons, although that’s atypical!).
What tools from your Ph.D. do you use most? Are there things you wish you had done more of as a student, or do you have any other advice for budding development economists thinking of jobs outside of a pure research track? Graduate school was definitely crammed with a lot of material I wasn’t interested in and knew I would never use. I actually didn’t even take development economics as a field because the course on offer at the time didn’t align with my interests. Honestly the only material from graduate school I’ve ever looked back at were from the first year econometrics course and the excellent applied micro labor course that David Lee gave during his short stint at Columbia. More generally I would say it’s important to build your econometrics, applied micro, and Stata skills as much as possible through coursework, research assistantships, and your own research. I was also lucky enough to spend a couple of graduate school summers in India working on a randomized evaluation with my adviser—field experience is definitely valued when we hire, so I would recommend seizing any of those opportunities. I do wish I had spent some time working on my foreign language skills when I had more free time as a student, as they have come in handy in my work in non-Anglophone countries.
DI: Please provide a short paragraph describing what you do in this job, and give us a sense of what a typical day or week might look like for you. My job is to conduct independent rigorous impact and performance evaluations of social programs in developing countries. Most of this work is conducted under contract to US government agencies (mostly MCC and USAID) and various foundations, who issue requests for proposals to evaluate their programs. In my eight years at Mathematica I’ve worked on evaluations in Asia, Africa, and Eastern Europe, and in topic areas including agriculture, primary education, vocational training, maternal and child health, land, and others. As senior researcher on an evaluation team I’m typically responsible for technical leadership of all aspects of an evaluation, including study design, data collection, and final analysis and reporting. Last week was fairly typical and included work on designing a randomized controlled trial of an anti-child labor program, drafting a quantitative survey of vocational education students, and planning the analysis of survey data from farmers in Morocco.
How much of your time do you spend a) doing research; b) reading research done by others; c) travelling in developing countries? I spend the vast majority of my time doing research, working on a handful of different evaluations at any given time. In terms of reading research by others, we typically do a formal literature review at the start of an evaluation to inform the study design (for example, statistical power calculations) and eventually to contextualize the findings. I also regularly read research conducted by other Mathematica teams as part of our internal quality assurance process, which is essentially internal peer review. Travel-wise, I travel to developing countries about four times a year, typically for a week or two per trip. These trips are short but really valuable in getting a taste of the local context and program implementation, as well as building relationships with local stakeholders. Besides the activities you mention I also typically work on a handful of proposal efforts each year to help us bid for new contracts, which typically last a few weeks each.
How easy is it for you to find time to take projects from the “deliver to client” stage to getting them through to peer-reviewed journal publication? I imagine you need to charge your time to specific grants/projects, and that once the report has been delivered to the client, it may be hard to then charge time over the next year or two to take it to the publication stage. We are definitely encouraged to publish our findings in peer-reviewed journals, and there are some internal resources available to work on publications if we can’t use contract funds (in some cases we can). The challenge is often just carving out the time to do the work, as one tends to quickly get busy on new projects. So in practice we probably don’t publish quite as much as we would like to, though it is a goal that we are working towards.
A related question is whether everything you do ends up in the public domain, or whether some of your evaluations are delivered to the client but then never released publicly?
Virtually all of our final reports are published on our website. In addition to reports, we also produce public or restricted use data files for some projects, which the client makes available for use by other researchers.
Tell us about one accomplishment in this job that you are particularly proud of. Although Mathematica has been working on international evaluations for more than a decade (and domestic evaluations since the 1960’s), our international research division was only founded a few years ago. It brought together staff from across the company who were passionate about international development into a single structure where we could collaborate more closely and expand our international work. I’m proud of the contribution that I—and all my international division colleagues—have made to the success of our division since it was established, by producing consistently high quality and informative research for our clients and stretching ourselves into new areas. The volume and scope of our international work today is really unrecognizable from when I started back in 2010.
How did you end up in this job? What do you like most about it, and what are the main downsides compared to being in a research university? Mathematica was actually my number one choice coming out of my econ PhD. My adviser had consulted for Mathematica on one of their first big international projects and told me about them; it sounded like a great opportunity to do something more practical with my PhD and avoid several years of stress worrying about tenure. In terms of the work itself, there are several similarities with academic research. I think the quality of our research is just as high from a technical perspective (we really care about getting the standard errors right!), which is important to me. Like academics, I also get to work on studies that interest me—we tend only to bid for work in the first place if we can form a team of interested staff. However, an important advantage is that I don’t have to “sell” my research to anyone as being interesting, as academics have to do. I think that it’s interesting simply because our clients are investing hundreds of thousands or even millions of dollars on these programs and need to know whether they work. I also really like my colleagues on a personal level, which makes for a very collaborative and supportive work environment. The only downside I can really think of compared to academia is less flexibility in one’s schedule, though Mathematica does offer a fair amount of flexibility relative to other private sector companies (I actually work remotely from Bangkok these days for family reasons, although that’s atypical!).
What tools from your Ph.D. do you use most? Are there things you wish you had done more of as a student, or do you have any other advice for budding development economists thinking of jobs outside of a pure research track? Graduate school was definitely crammed with a lot of material I wasn’t interested in and knew I would never use. I actually didn’t even take development economics as a field because the course on offer at the time didn’t align with my interests. Honestly the only material from graduate school I’ve ever looked back at were from the first year econometrics course and the excellent applied micro labor course that David Lee gave during his short stint at Columbia. More generally I would say it’s important to build your econometrics, applied micro, and Stata skills as much as possible through coursework, research assistantships, and your own research. I was also lucky enough to spend a couple of graduate school summers in India working on a randomized evaluation with my adviser—field experience is definitely valued when we hire, so I would recommend seizing any of those opportunities. I do wish I had spent some time working on my foreign language skills when I had more free time as a student, as they have come in handy in my work in non-Anglophone countries.
Join the Conversation