I have started reading K McGillivray, Government Cloud Procurement. Contracts, Data Protection, and the Quest for Compliance (Cambridge University Press 2022), which promises to be a big addition to the literature on the procurement of digital technologies. One of the key issues the book explores at length is the central role that public contracts play in filling (some of the) regulatory gaps left by the absence of legislation addressing the challenges of cloud computing.
This got me thinking that this gap-filling function of public contracts in the cloud sphere is reflective of the broader role that procurement procedures and the ensuing public contracts are starting to develop in relation to other types of digital technology—notably, artificial intelligence (AI).
Procurement regulation will increasingly (be expected to) play a crucial gatekeeping role in the adoption of digital technologies for public governance and public service delivery. As rightly stressed: ‘The rules governing the acquisition of algorithmic systems by governments and public agencies are an important point of intervention in ensuring their accountable use’ [Ada Lovelace Institute, AI Now Institute and Open Government Partnership, Algorithmic Accountability for the Public Sector (August 2021) 33]. Ultimately, contracts and other arrangements for the development and acquisition of digital solutions are the entry point into the public sector for these innovations, and the procurement rules can be either a catalyst or a hindrance to co-production and experimentation with digital governance solutions.
The gatekeeping role of procurement underpinned eg one of the recommendations of the UK’s Committee on Standards in Public Life, in its report on Artificial Intelligence and Public Standards: ‘Government should use its purchasing power in the market to set procurement requirements that ensure that private companies developing AI solutions for the public sector appropriately address public standards. This should be achieved by ensuring provisions for ethical standards are considered early in the procurement process and explicitly written into tenders and contractual arrangements’ (2020: 51). A variation of the gatekeeping approach can concentrate on procurement practice and the embedding of specific controls as a matter of public buyer deontology [see P Oluka Nagitta et al., ‘Human-centered artificial intelligence for the public sector: The gate keeping role of the public procurement professional’ (2022) 200 Procedia Computer Science 1084-1092].
There is thus a growing recognition of the pragmatic utility of leveraging procurement mechanisms to ensure transparency and accountability in algorithmic systems, particularly considering that these systems play a crucial role in policymaking and decision-making by public agencies [DK Mulligan and KA Bamberger, ‘Procurement as policy: Administrative process for machine learning’ (2019) 34(3) Berkeley Technology L. J. 773-851]. Consequently, there is increasing interest in a reassessment of the existing procurement rules as they apply to contracts for digital technologies; as well as in the redesign of procurement to foster reliability, sustainability, and public trust in AI [see e.g. UK Government, BEIS, DCMS and Office for AI, Guidelines for AI procurement (8 June 2020); also W Naudé and N Dimitri, ‘Public Procurement and Innovation for Human-Centered Artificial Intelligence’ (2021)].
However, the challenges in effectively mobilising procurement for this gatekeeping function are yet to be properly conceptualised and understood [See e.g. P Nowicki, ‘Deus Ex Machina?: Some Remarks on Public Procurement in the Second Machine Age’ (2020) 1 EPPPL 53-60; see also K Mcbride et al, ‘Towards a Systematic Understanding on the Challenges of Procuring Artificial Intelligence in the Public Sector’ (2021)].
As I keep thinking about this (see here for earlier thoughts), I am realising that the emerging discussion or conceptualisation of public procurement (or procurement professionals) as gatekeepers of the adoption of AI by the public sector (and more broadly) can fall into the same trap of partiality as the equivalent discussion of financial gatekeepers in the corporate governance sphere years ago.
As Prof Coffee brightly pointed out [Gatekeepers: The Professions and Corporate Governance (OUP, 2006) 2-3] in the context of financial markets, there are two important dimensions of gatekeeping at play: one concerns ‘strict’ gatekeeping in terms of veto capacity (eg an audit firm can decline providing an opinion on corporate accounts, or a lawyer can decline to provide an opinion required for the closing of a specific corporate transaction). The other dimension, however, concerns a reputational aspect of gatekeeping that can generate reliance by third parties (eg an investment bank acquiring shares of a target company can lead others to also invest in that company).
In the procurement context, it seems to me that there is also a strict gatekeeping function (procurement requirements determine which technology/provider cannot get a public contract, eg to protect a specific public interest or avoid a specific public harm; or which one can provided it abides by specific contractualised requirements), as well as a reputational gatekeeping function (eg procurement of specific technologies/from specific providers can have a signalling effect that triggers further procurement by other public buyers and/or adoption by the private sector).
While in financial markets the reputational aspect is dependent on market-based issues (such as repeat transactions), in procurement settings reputation is almost a given due to a presumption of strict scrutiny of public providers (and thus the importance of ‘past performance’, or other signals such as being able to disclose that a technology or provider is used by Department X, or in some other settings ‘by appointment to HM the Queen’). This compounds the importance of procurement gatekeeping, as it not only concerns the specific decision adopted by a given public buyer, but also the broader access of technologies and providers into the public sector (and beyond).
However, a significant difference between gatekeeping in financial markets and in procurement however stems from the likely main source of potential failure of the gatekeeper. While in financial markets gatekeepers can be expected to be high-skilled but subject to structural conflicts of interest, in particular due to the way they are remunerated (which impinges on their independence), in procurement markets there is a real risk that public buyers are not only subject to potential conflicts of interest (an enduring issue in procurement regulation, and the source of incomplete attempts at the regulation of conflicts of interest and integrity in procurement), but also underprepared for the gatekeeping task.
In other words, the asymmetry of information seems to operate in reverse in both settings. While in financial markets the superior information and related skills belong to the gatekeeper (as compared to the retail, or even (passive) institutional investors), in procurement markets the information and skills disadvantage plays against the gatekeeper (public buyer) and in favour of those seeking to sell their technology.
And this is where the analysis by McGillivray is again interesting, as it highlights compliance challenges and gaps resulting from the parallel procurement-based gatekeeping of data protection law in the government cloud procurement sphere. Plenty food for thought (at least for me).