Along with the Center for Experimental Social Science at Nuffield College at Oxford, eMBeD co-organized a conference called “Measuring the Tricky Things.” The lineup included Susan Fiske presenting a magisterial overview of her decades-long work on the stereotype content model, Armin Falk on his groundbreaking study of time, risk, and social preferences among 80,000 individuals in 65 countries, Karla Hoff on using lab in field experiments to identify the honor ethic among higher caste villagers in North India, Ryan Enos on measuring racial attitudes, Rachel Glennerster on measuring women’s empowerment, Julian Jamison on how and why to use item count techniques to mitigate social desirability bias, Henry Travers on debiasing estimates of wildlife survival, Amandi Mani on assessing the effect of financial worry on cognitive performance with cell phones, and Sheheryar Banuri on using videos to probe the effect of pro-poor bonuses on doctor’s decisions on which patients to see. My eMBeD co-head Renos Vakis assessed the strengths and weaknesses of World Bank surveys on socio-emotional skills. I discussed the reliability and validity of measurements of social norms with respect to women’s labor force participation in Jordan.
To summarize, Raymond Duchy of Nuffield said measurements in the economic and social sciences can be tricky for three reasons. We can measure a previously studied variable with a new modality (cell phones to assess cognitive ability, pre-recorded videos to measure the pro- or anti-poor biases of health care workers). We can measure a new variable or refine the conceptualization of previously studied variable (concepts and methods for measuring social norms, empowerment, racial biases, and socio-emotional skills; validated measures of preferences; techniques to take out social desirability bias). Finally, we can measure known or new variables in new or challenging contexts (selecting the best measures of gender norms or women’s empowerment in a particular country, creating a dataset of globally comparable preference measures, or applying a variable to a new subject areas).
What I left thinking about, however, was how measuring the tricky things can change our frameworks. Fiske’s evidence shows that while stereotypes can change over the years, they are also relatively stable. Why? Seeing one example of a racially marginal group doesn’t by itself change prejudices because people are cognitively lazy – they won’t update their views unless the group’s overall status position changes in society. A cognitive process, perhaps something like the representativeness heuristic, affects social preferences. Relatedly, Falk’s data show that social preferences, including reciprocity, are correlated with risk preferences, and that (self-reported) math skills, a proxy for intelligence, are correlated with all of the preferences Falk measures.
What we think of the as the social and cognitive domains may not be as easily separable as we think. In the WDR 2015, we developed a framework of thinking automatically, thinking socially, and thinking with mental models. It’s still a useful framework, but it’s also likely the case that when we revisit the framework some years from now, automatic and social thinking may arise from a set of more fundamental and common processes.
Human beings are tricky things.
Join the Conversation