Syndicate content

Add new comment

Submitted by BillWarburton on

Transparent assignment rather than random assignment.
I strongly agree that the road to better evaluations lies on the path of determining the most ethical way to allocate scarce resources. As Keynes pointed out in 1937 (1), and as everyone who has worked in government realizes, “There is nothing a government hates more than to be well-informed; for it makes the process of arriving at decisions much more complicated and difficult.” In addition program administrators are naturally hostile to evaluation—the best outcome that they can hope for from an evaluation is that it confirms the rosy picture that they have painted of their program. The worst is that their program will be found to be useless and their budget is cut.

The strong forces that oppose (rigorous) evaluation will often employ ethical arguments against random assignment (2) and in the absence of random assignment the default is an unknown selection mechanism. By shifting the debate from “Random assignment or not?” to “What is the most ethical method of selection?”, we avoid the unknown selection mechanism, the alternative that precludes rigorous evaluation.

In some cases, as David McKenzie points out (March 19th) randomization will be found to be the most ethical method of allocating scarce resources. In other cases we may be advised to allocate resources to those who most need it first (with a Rawlsian justification). Some would argue that that outcome would be preferred by an evaluator since the Regression Discontinuity Design estimates would give us a better understanding the marginal benefit from expanding a program than we would be if we had used random assignment. But whatever method is found to be most ethical will allow us to produce unbiased estimates of impact as long as the selection method is transparent.

By definition, making the proper ethical choice is the right thing to do. IMHO paradoxically, failing to explicitly put the ethical argument first gives those who oppose evaluation for selfish purposes a powerful rhetorical tool, and in the final analysis reduces the probability of a successful evaluation.

(1) JM Keynes. The Times (March 11, 1937); Collected Writings, vol. 21, p. 409.