Some advice from survey implementers: Part 2

This is part 2 of a two part blog on what survey implementers would tell the researchers and others who work with them (part one is here).   Before we dive in, I want to reiterate my thank to the folks at EDI and IPA, as well as James Mewera of the Invest in Knowledge Initiative, Ben Watkins at Kimetrica,  and Firman Witoelar at SurveyMeter who took the time to send me really careful thoughts and then answer my queries.   As before, don’t take anything below as something specific any one of them said – I’ve edited, adjusted and merged.   Blame me if you don’t like it.  One final note, as you can see from the list, not everyone one of these is a commercial firm, and some of them do research as well – so not only keep that in mind when filtering the advice, but I’ll abbreviate with SO for survey organization.     
 
Please read this post as me channeling and interpreting their voices.  I am not sure I agree with everything I heard, but I am passing it on.   And all of it gave me food for thought.   Stuff in [italics] is me explicitly responding to a couple of points.   
 

Some general advice for when things are in progress

• Don’t dither too much.  Your indecision has a cost for the SO as they are paying people while everyone waits for you to make up your mind.  
• In that vein, respond promptly – since the meter is running, these are emails that should jump to the top of the list.
• Visit the field.   This is particularly important during piloting and training – both for the SO asking you questions and you understanding what is going on. 
• When you are visiting the field, try not to micromanage.   As one example, if you are going to correct an enumerator mid-interview, step back and talk to the SO management about the best way to communicate this error to the entire field team – not just the one you are working with. 
• Set up a fieldwork monitoring and reporting plan with your SO. 
• You are highly likely to have to compromise on something. So be prepared.  
• You might not understand the norms in dealing with government, implementers, and/or stakeholders.   This will make things difficult for everyone.   Here a local partner is key, be they a co-investigator, research assistant or some other kind of liaison person. 
• Don’t move the goal posts after you sign the contract or surprise your SO with things that you didn’t put in the original terms of reference (if you can help it).  Make those original terms of reference as explicit as you can. 

• It seems obvious, but define roles and responsibilities early on and, if you are going to be an absentee researcher, make sure there is enough labor at the right level to cover your absence.  
• Set up clear expectations and norms about how, when, and about what you will communicate from the start.
• Keep tabs on the project finances.   This is going to be a collaborative effort with the SO.  And when the project changes direction, don’t expect it to be free.  Document these discussions so you all have something to go back to.  
• Keep in mind that if the money is going straight to the SO, or they were contracted by your projects office or procurement, they’re going to be bound by certain obligations and contractual provisions.
• If you want particular staff on your survey, be prepared to pay extra.   If you know these folks are good, you’re probably not alone.    Another option is to think of ways to guarantee them longer term employment than those you will be competing with.    
• If you are bringing in international staff for parts of your survey, think about hiring some local staff to be their understudies.   Not only are you building capacity, you might need it when you do your next survey.    

• Keep in mind that when you are interfacing with an research assistant employed by the SO, they’re also being managed by someone else.   And yes, when things are going well, that manager deserves some credit too.
• Invest in that research assistant early.  It will pay off. 

• More than one SO made the point to align incentives.  If you (the researcher) and the SO are the same team, a lot more will line up.   Some SOs will even help with raising additional funds if needed.  
• One issue that came up a couple of times was thinking about (potentially) including someone from the SO as a coauthor.   I’ve seen various standards for co-authorship, so this is something to think about.  
• At the very least, you do want to think about the SO for the acknowledgements in the paper either as an organization and/or particular people within the organization who really helped you out.     While you are doing that, you shouldn’t be shy to reach out to them with questions or for advice as you are writing.  
• Keep in mind that the SO could have institutional memory that you don’t.   Consider this anecdote:   “What is often under-appreciated is that SOs usually play a huge role in shaping the survey instruments that ended up in the field. This is true especially for SOs that have been around for a long time.  They use what they learn from all the surveys that they have done to provide feedback to researchers. A lot of time this contribution is not acknowledged in project reports or papers. Once we received a draft questionnaire from researchers who asked us to try it. It turns out that it came from an earlier project of ours, with all of the typos and grammatical errors still intact.”
• Think about local dissemination not only for policy relevance, but also for your SO (and your future survey plans). As one SO writes:“researchers usually prefer to disseminate finding in external forums. This is much appreciated but it does not do good to the local players. It puts a survey implementer in an awkward position when local stakeholders accuse the implementer of conducting studies without sharing any results. Local partners/stakeholders would say to the implementers, ‘you are the guys that come for permission to do research and you disappear into thin air only to reappear looking for permission to conduct another study.’ Of course, researchers own the data but should always strive to honor the local players by informing them about the results. Implementers have relationships to nurture and reputation to protect.”
• Local officials can hassle your survey teams (e.g. imposing arbitrary conditions, asking the SO to hire their relatives or favorites).  Sometimes, the SO may need you to intervene as you might carry more weight than they do.