A colleague stopped me by the elevators while I was leaving the office.
“Do you know of any paper on (some complicated adjustment) of standard errors?”
I tried to remember, but nothing came to mind – “No, why do you need it?”
“A reviewer is asking for a correction.”
I mechanically took off my glasses and started to rub my eyes – “But it will make no difference. And even if it does, wouldn’t it be trivial compared to the other errors in your data?”
“Yes, I know. But I can’t control those other errors, so I’m doing my best I can, where I can.”
This happens again and again — how many times have I been in his shoes? In my previous life as an applied micro-economist, I was happily delegating control of data quality to “survey professionals” — national statistical offices or international organizations involved in data collection, without much interest in looking at the nitty-gritty details of how those data were collected. It was only after I got directly involved in survey work that I realized the extent to which data quality is affected by myriad extrinsic factors, from the technical (survey standards, protocols, methodology) to the practical (a surprise rainstorm, buggy software, broken equipment) to the contextual (the credentials and incentives of the interviewers, proper training and piloting), and a universe of other factors which are obvious to data producers but usually obscure and typically hidden from data users.