Quite a few years ago I worked on an impact evaluation with a government, in which the government was paying directly for the baseline survey. Rather than calling multiple survey firms, discussing carefully their experience, interviewing them on how they would deal with issues we wanted addressed, getting and negotiating prices, etc with them, we instead had to go through the government’s procurement process. This meant trying to specify ex ante all the many details of the survey, many of which were uncertain and would only be known once the research was underway, and specifying quality criteria that survey firms had to meet. These criteria could not be too stringent we were told, since the government needed there to be multiple bidders. Their procurement process then awarded the contract to the lowest priced firm among those that met the quality threshold. Sound reasonable? Perhaps for some types of products, but for contracting experimental survey work this was “not great”. The survey data had several irregularities in strange ways that we couldn’t have ex ante contracted on, and we ended up having to fundraise to pay directly for all follow-up surveys ourselves, contracting a different firm.
So why do governments use such processes, instead of allowing some discretion. The main reason is concerns that contracts that are not awarded subject to rules-based competitive bidding will end up costing more money, and potentially be awarded by favoritism, or through corrupt practices such as kickbacks. So most governments (and World Bank contracting) have strict procurement processes with competitive bidding for many contracts. A nice working paper by Dimas Fazio examines whether there is a trade-off between price efficiency and quality in public procurement, and shows some rationale for discretion.
The setting: Public procurement in Brazil
The paper uses a database of all federal public procurements in Brazil, and focuses on procurements for purchasing goods, so that it can use brand as a measure of quality. In total it uses 2,506,809 distinct procurements, purchasing 57,168 products from 81,445 firms for 4,657 federal and local government agencies from January 2013 to December 2020. For example, procurement of products like printer cartridges, ballpoint pens, coffee, sugar, surgical masks, microscopes, and essential ER drugs like insulin.
The two main procurement procedures are (1) bid waivers, which allow government agencies to procure items without competitive bidding when the value is below a threshold (or in case of emergencies). This threshold was 8,000 BRL (just over US$2,000 at the time), extended to 17,600 BRL in 2018 (around US$4,800); and (2) competitive bidding via auctions, where the lowest price wins. (see this lovely paper by Ferraz et al. which looks at the impact of narrowly winning the auction on SME firm growth). One can then compare the prices and quality of what is procured on either side of this threshold.
· Government agencies like discretion/there is bunching just below the threshold: the first result comes from looking at the distribution of the value of procurement around the small value exemption threshold. There are about 17 times more procurements below the SVP threshold than above it, and about 5 times more procurements than a predicted counterfactual in the absence of bunching. One way agencies seem to achieve this is by fragmenting their purchases – e.g. by splitting the procurement of similar products into multiple purchases, rather than bundling together as one transaction.
· Discretion does result in higher prices, but half of this reflects higher quality. Figure 4 from the paper, shown below, compares prices of products purchased around the threshold. Panel A shows considerably lower prices for the same product purchased via competitive auction than in the small value discretionary procurement. Running an RDD-type regression, prices are 23% higher for products procured via discretion. However, panel B shows that this difference becomes smaller once one controls for brand as well as product, with the price difference about 12%. Using consumer technical quality review data, the author can show these higher prices brands are indeed better quality. Moreover, as Figure 4 shows, the prices rise sharply just before the threshold, which appears to reflect the strategic sorting of agencies around the threshold. Doing a donut-hole type analysis which drops procurements in a hole right around the threshold reduces the price differences down to around 6% after controlling for quality, and to only 3% when comparing discretion to auctions that have below the median number of suppliers participating.
When is better quality desirable?
So there does appear to be a trade-off: allowing discretion in procurement does result in higher prices being paid, and the firms that get the contracts are more politically connected. However, they are also more likely to be nearby, larger, and more established, and better able to provide higher quality products. Now, perhaps it is not worth government agencies paying for higher quality coffee or sugar, but it may be worth paying for quality of certain types of inputs. The question is whether allowing for discretion allows for better delivery of public services.
As supportive evidence that it can matter, the author uses data on hospital deaths from 63 public hospitals in Brazil and matches this with procurement data. Data are at the hospital-cause of death-quarter of the year level. He then can control for a variety of fixed effects (hospital-cause, cause-quarter, and state-quarter or hospital-quarter), and finds a significant negative association between the share of procurement on essential drugs for a given cause that are procured below the threshold (using discretion) in a given hospital and quarter, and deaths of that cause in that hospital and quarter. That is, using discretion appears to enable hospitals to buy better quality drugs and prevent deaths. This is strongest for deaths from preventable causes, with no impact on cancer deaths. This is good suggestive evidence, but it is unclear what causes the share of drugs for a given cause that are procured via discretion to vary over time within the same hospital – and whether we can truly treat it as random conditional on all the controls added.
Discretion may matter even more for services
The paper focuses on product procurement, where quality is much easier to ascertain. One simple solution would be to enable the procurement auctions to specify quality as well as product (e.g. we want to procure this much insulin of this brand). Perhaps the current procurement rules prevent this, or perhaps the people in charge of procuring items do not know is out there on the market, and rely on suppliers to suggest quality. But either way, product quality is much more observable, the product is either delivered or not delivered, and it can even be returned if not satisfactory.
In contrast, procuring services is much, much harder in many cases, and it is often impossible to completely contract in advance on all the things that need to be done. Government procurement contracts for construction are notorious for having firms bid with prices and delivery dates that then get revised multiple times over time (I’m still waiting for that Silver Line extension to Dulles). Similarly, in working on research processes, where you learn what you need as you go, and have to revise things frequently, it becomes much more like a partnership than buying a box of ballpoint pens at cheapest cost, and you need someone you can trust and work together with. Of course it is much harder to research the impacts of different procurement processes on services, given the difficulties of measuring quality, but I would love to see a paper that builds on this one and looks at services.
Join the Conversation