My colleague and (I hope still) friend, Chico Ferreira recently took the trouble to write a comment on my earlier LTD post on measuring inequality of opportunity in the context of human development. Early on in his comment, Chico also paid me the compliment of a being a “clever guy”, which was nice until I read on and found that while he agreed with some of what I said there was a lot he didn’t like. Now Chico is a really clever guy, and this is an area he knows a lot about. So I realize I’m treading on thin ice when I say I’m not completely convinced about his ripostes. But let me take the risk. Chico’s not just super-clever – he’s also very nice. So if the ice cracks and I fall in, I think there’s a good chance he’ll pull me out.
My first quibble was with the human opportunity index (HOI) on the grounds it focused on that part of inequalities in childhood outcomes that could be attributable to inequalities in circumstances. Chico says the reason I object to this approach is because “in Adam’s view, all of that inequality is unfair”.
Not quite, Chico. It’s the LAC report – which you co-authored – that says that all childhood inequalities are unfair. On p3 of the report you and your co-authors say: “for children, access defines ‘opportunity,’ because children (unlike adults) cannot be expected to make the efforts needed to access these basic goods by themselves.” I happen to agree with you, but that’s beside the point. My point is that if your reports starts with the premise that all childhood inequalities are unfair because children can’t be held responsible for their parents’ lack of effort in getting them immunized, getting them access to safe water, etc., then what you should have measured is inequality in outcomes – not the part of the inequality that’s due to inequality in circumstances.
Apparently I’m not the first to say this. You say in your comment: “That this point had been made many times before does not detract from its value, and it is incumbent upon users and promoters of the HOI to be explicit about this interpretational limitation.”
I’m not sure quite what to make of this. If you believed your premise, the HOI is quite simply the wrong measure of inequality. It seems you’re acknowledging this. In which case isn’t it incumbent on you and your co-authors to come out and say point blank: “Sorry, folks, we misled you. Please don’t use the HOI with kids. It measures only part of what the inequality we believe to be unfair and possibly a very small part. Please instead use a standard inequality measure because all inequality among children is unfair.” I’m probably missing something here, and suspect the ice might crack any minute.
I know I’m skating on really thin ice on my second point which concerns the index of economic opportunity (IEO). This gets applied to teenagers (educational achievement among 15-year olds) and adults (income, consumption and earnings); these are, according to Chico and his co-authors, people we should be able to hold accountable for their lack of effort. The ice is getting super-thin here, because Chico dreamt up the IEO and has spent a lot of time working on it. So let me tread very carefully.
Chico acknowledges that the IEO is a lower bound estimate of inequality in opportunity. He points out that there are two groups of things he can’t observe in his data. There are unobservable factors that are not beyond the control of the individual, like effort: inequalities in these shouldn’t be counted as inequality of opportunity according to Chico, so their exclusion doesn’t matter. But there are also unobservable factors that are beyond the control of the individual, and these ought to be counted as part of inequality of opportunity. Chico gives the example of a teenager who feels sick on the test date and underperforms. If Chico can’t observe the teenager’s sickness on the test day, he’s forced to conclude from the data that the youth had the opportunity to do well on the test, but chose not to. Some inequality that ought to be counted as inequality in opportunity will get mislabeled as inequality. Chico justifies plowing ahead despite this mislabeling on the grounds that countries vary in his IEO index, and vary too in the share of inequality he ends up labeling as inequality in opportunity. The IEO index also correlates across countries with things it might be expected to correlate with such as intergenerational mobility.
I had a couple of reactions when reading Chico’s defense.
First, I think Chico risks mislabeling a lot more inequality than he’s owning up to. Mislabeling can arise not just from factors that can’t be observed in the dataset but also from factors that are in the dataset but are unjustifiably excluded from the set of variables used to define circumstances. The LAC report includes just five factors capturing circumstances: gender, race or ethnicity, birthplace, educational attainment of the father, and educational attainment of the mother. As I said in my post, the educational achievement of a 15-year old reflects a lot of decisions over the previous 14 years over which the teenager had little if any control, including the family’s living standards. To be sure the constraints on the teenager’s educational performance at age 15 will reflect the five factors Chico includes. But they’ll also reflect choices the kid’s parents made, as well as a host of factors beyond the control of the parents and the kid. Teenagers may be able to decide how much effort to put into their homework, but the list of “exogenous” constraints on their performance goes well beyond gender, ethnicity, birthplace, educational attainment of the father, and educational attainment of the mother. At the minimum there seems to be a case for exploring the implications for the IEO of including a much longer list of circumstance variables.
Second, even with a more reasonable list of circumstance variables, we’ll still end up a long way from where we should. There are just so many things we can’t hope to capture in surveys that influence outcomes (especially test scores) but ought not be misclassified as “effort”. My guess is (there’s doubtless a literature on this) that differences between teenagers in test scores has less to do with their effort in the classroom than with differences in their school and home environments. I suspect luck also plays a big part, even controlling for school and home environment. Luck certainly plays a big part in shaping health outcomes. As I said in my post, for teenager test scores and for all health outcomes, I suspect a lot of people would happily simply label all inequality as unfair.
Chico says of the IEO “if properly understood, this information can be as useful as other common scalar measures we often come across like the headcount index, the poverty gap, or the Gini coefficient”. I’m not so sure. I may be wrong but I presume we can’t say how far the IEO of a specific country at a specific date falls short of its true value. Couldn’t countries have the same true value but have different estimated values, and vice versa? Couldn’t a country’s estimated value fall over time without its true value falling? Presumably the answer must be Yes. In which case, I’m not quite sure how to “properly understand” the IEO. And it surely marks out the IEO as different from a poverty or inequality number. There may be data issues that make our poverty and inequality estimates problematic, but they don’t, as far as I know, lead us to systematically underestimate what we’re trying to measure by an inherently unknown amount.