A recently published working paper has assessed the impact of increased availability of procurement data on competition for public contracts and on procurement outcomes in the EU context: R Duguay, T Rauter & D Samuels, ‘The Impact of Open Data on Public Procurement’ (November 22, 2019).
Duguay, Rauter & Samuels concentrate on the increased availability of TED data in a (more) user-friendly format in July 2015 (when the data started being available for a bulk download on CSV format) to assess the effects that easier access to procurement data has on the functioning of procurement markets and on procurement outcomes. The paper is very interesting and their results are quite striking.
However, it is important to stress the important caveat that their analysis is still based on TED data and, thus, potentially affected by the quality shortcomings of that data. As mentioned in other occasions, the TED database has problems because it is constructed as a result of the self-declaration of data by the contracting authorities of the Member States, which makes its content very inhomogeneous and difficult to analyse, including significant problems of under-inclusiveness, definitional fuzziness and the lack of filtering of errors—as recognised, repeatedly, in the methodology underpinning the SMSPP itself (see here and here).
With that in mind, however, it is interesting to look closely at their findings.
A seemingly striking insight derived from the paper is that ‘the new European government contracting provisions have anti-competitive effects‘ (at 17). This is in the context of an analysis of the ‘likelihood that government agencies allocate public contracts through an open procedure‘ and should thus not be surprising, given the flexibilisation in the use of procedures involving negotiations. However, even with this regulatory effect, the authors find that more open data triggers more use of open procedures, in particular in EU countries with weaker institutional frameworks (at 18-19, and see below). This could be symptomatic of the fact that more complexity in procurement subjected to higher levels of transparency pushes for a risk-averse approach to procurement compliance. The same would be supported by their finding of higher levels of award of contracts on the basis of price-only award criteria (at 25, and see below).
This tension between procurement complexity and transparency is generally strongly evidenced in the paper.
On the one hand, and in line with claims of the pro-competitive nature of more openness in procurement data (note, not of more openness or transparency of contract opportunities), the authors find that
the likelihood of competitive bidding increases sharply for TED contracts around July 2015 and that this increase persists through the end of our sample period [ie to the end of 2018] (at 18);
open procurement data leads government officials to implement more competitive bidding processes [ie open procedures], and that this increase in competitive bidding is driven by countries that do not have the institutions to effectively monitor public officials (at 19);
the number of bids increases sharply for TED contracts soon after the open data initiative, and this increase persists throughout our sample period (at 20);
public officials are 8.7 percentage points more likely to award government contracts to new vendors after the open data initiative (at 21);
contract values fall by approximately 8% ... after the open data initiative (at 23).
On the other hand, and also in line with theoretical expectations of a degradation of procurement decisions subjected to higher levels of transparency (and the fact that this transparency does not concern contract opportunities, but more general open procurement data), the authors also find that
[the results] are inconsistent with the idea that easier access to procurement data fosters cross-border competition throughout the European Union … open procurement data fosters local competition among vendors by reducing barriers to entry but does not promote cross-border competition across the European single market (at 22);
after the open data initiative, the likelihood of a contract modification increases by 2.9 percentage points for contracts above TED publication thresholds (at 24);
after the open data initiative, public officials are 38% ... more likely to award contracts above TED publication thresholds exclusively based on price (at 25);
the performance ... is significantly worse if price was the only award criterion in the allocation decision (at 26);
the increase in modifications is driven by contracts awarded to new government suppliers, consistent with information asymmetries contributing to the observed deterioration in contract performance. Moreover, this evidence suggests that procurement relationships before the open data initiative were not necessarily corrupt or otherwise inefficient (at 26);
the decline in contract performance is stronger for complex procurements, consistent with project complexity exacerbating the potential allocative distortions of open procurement data (at 27).
Their overall conclusion is that
Comparing government contracts above and below EU publication thresholds, we find that increasing the public accessibility of procurement data raises the likelihood of having competitive bidding processes, increases the number of bids per contract, and facilitates market entry by new vendors. After the open data initiative, procurement prices decrease and EU government agencies are more likely to award contracts to the lowest bidder. However, the increased competition comes at the expense of lower contract performance, particularly if suppliers are new, procurement projects are complex, and contracts are awarded solely based on price.
Overall, our results suggest that open data on procurement awards facilitates competition and lowers ex-ante procurement prices, but does not necessarily increase allocative efficiency in government contracting (at 27-28, emphases added).
I find these results striking and difficult to assess from the perspective of evidence-based policy-making. There are two issues of particular concern/interest to me.
One, the finding that more availability of data does not generate more cross-border procurement, and that the push for more competitive (ie open) procedures is mostly appreciable in countries with weaker institutional frameworks. This could support the position that institutional robustness is an alternative to data transparency, which would significantly alter the prioritisation of systemic procurement reforms and take the sides of systems that favour strong institutional oversight in a context of relative opacity.
Second, that transparency exacerbates problems at execution phase, in particular in complex projects and/or projects with new suppliers. This would take the wind out of the sails of reform and policy-making approaches concentrating on perceived or apparent competition for the contract at award stage, and rather force a refocus on an analysis of procurement outcomes at the end of the relevant project. This would also side with approaches that would advocate for more robust institutional approaches to contract design and performance management, rather than relying on transparency to correct contract execution problems.
The mixed results of the paper are also interesting in the context of the long-term effect of more open procurement data on competition, as well as on cartelisation and bid rigging risks, which are not assessed in the paper.
On the round, I think that the paper offers some interesting evidence to back up that there is a need to reconsider the level of transparency given to procurement data. I do not think this should stop the development of an improved procurement data architecture in the EU. To the contrary. I think this should reignite and prioritise discussions concerning the level of disclosure or public access to that information (ie its openness), which cannot be simply assumed to be positive in what, in my view, is currently an excessively simplistic approach in leading policy-making and think tank proposals. For more (but not new) discussion, see here and here.