Jamillah Knowles & Digit / https://betterimagesofai.org / https://creativecommons.org/licenses/by/4.0/
If we live in the age of artificial intelligence, the proposed EU Public Procurement Act is anachronistic in its omission of AI. AI simply does not feature in the Public Procurement Act — or at best, only as an after-thought or ‘something to think about at a later time’. Given the continuous expectation that procurement will operate as a key governance tool for public sector AI use, that is not good enough.
As is now trite, there are two major interactions between AI and procurement: procurement is the gateway for AI acquisition by the public sector (‘AI procurement’); and AI can be (and is increasingly) used to manage procurement (‘AI for procurement’). Here, we can distinguish the direct use of AI for procurement and the use of AI to monitor procurement (eg to tackle corruption or collusion risks). The latter could be thought about as a meta application of AI for procurement.
Despite the growing understanding of these two/three layers of issues and the emergence of a quickly-expanding academic literature, as well as policy and guidance literature from across a wide variety of jurisdictions, the Public Procurement Act does not include meaningful consideration of AI.
Missing priority
To be fair, other than in relation to some meta applications of AI for procurement, AI also did not feature in the key documents influencing the Commission’s approach to the proposal. The report by the European Court of Auditors and the Council’s conclusions that kick-started the review solely referred to AI in passing (in relation to automated translation by ECA, and about exploitation of public procurement data by the the Council). The Parliament’s resolution on procurement similarly solely stressed that the transition to a digital-first approach would “reinforce the possibility of public scrutiny in the procurement process, especially considering the potential offered by disruptive technologies such as artificial intelligence” and called the Commission to update its tools for monitoring competition in public procurement. As far as I know, only the opinion of the European Committee of the Regions has gone beyond consideration of these meta uses of AI for procurement, and stressed that “it is necessary to provide a specific legal regime for the use of AI in the area of public procurement, thereby guaranteeing a framework of legal certainty” (para 38).
Similar considerations were captured in the public consultation underpinning the proposal. As the Commission’s impact assessment reflects, “several respondents see AI and advanced analytics as a way to improve procurement governance, through automated checks, risk scoring, fraud detection, and carbon-footprint verification. At the same time, they call for clear rules on the use of AI and algorithmic tools in award processes, to maintain transparency, fairness and legal certainty”.
The Public Procurement Act’s approach
This has not translated into any provision in the proposed Public Procurement Act, not even in relation to the (meta) use of AI for procurement. All there is, in the Legislative financial and digital statement (part 4.3), is a consideration that several elements of the envisaged new digital architecture (such as the Commission’s eProcurement platform, the electronic eligibility service, or the procurement data spaces) could interact with the EU AI Act. However, there is a repeated statement that “No AI-based decision-making is foreseen for the core legal obligations. If automated risk detection, ranking, recommendation or fraud analytics are introduced, an AI Act risk/compliance assessment will be carried out.” So, if anything, the proposal suggests that the digital architecture will not include ways to embed AI for the purposes of any of the key aspects of a procurement procedure (ie ‘core elements’). Is this a tenable position? Is this a ‘future-proof’ approach to regulating procurement in the EU, especially on the basis of a single rulebook? What would happen when Member States start (or continue) deploying AI in relation to ‘core obligations’? Would that be permitted? Under which conditions? How would divergence in the (applicable by default) administrative law of the Member States be negotiated?
Regulatory assumptions?
More broadly, it seems that, at best, the Commission operates on the assumption that the EU AI Act would provide the regulatory architecture for the use of AI in procurement (if there was any), but this is only explicit in relation to the meta use of AI for procurement and pending some future impact assessments (not least, because there is no detail whatsoever on what those components will look like in the many delegated and implementing acts required to develop them). Even then, the use of AI for procurement is not really considered in full, such as eg the Public Procurement Act’s omission of the more immediate potential obligation to disclose human-AI interaction under the transparency requirements of Art 50(1) EU AI Act even where no AI-based decision-making takes place—as soon as tenderers or public buyers eg interact with a chatbot.
The procurement of AI is, in itself, not even referred to in the proposal. Perhaps this is based on the Commission’s view that its (meagre) work on this issue to date suffices.
However, this ‘silent approach’ to reflecting the AI/procurement interactions in the Public Procurement Act ignores the fact that, other than transparency, the EU AI Act does not create obligations for the use of AI in procurement and, at best, provides a very weak level of regulatory assurance for AI procurement, and that the model clauses for AI procurement are nothing but an empty shell.
taking (more meaningful) action
All in all, the Public Procurement Act does not contain any provisions on AI procurement, does not fully consider the use of AI for procurement, and only hints at the future consideration of meta uses of AI for procurement monitoring purposes. This is at odds with the pressure for procurement to regulate AI acquisition by the public sector, as well as the pressure to deploy AI to manage procurement processes.
In my view, there are at least three improvements the co-legislators should introduce in the legislative process the Public Procurement Act is undergoing.
First, the Public Procurement Act should reform the EU AI Act and, in particular, Annex III to include the use of AI for public procurement (both directly, and in relation to its meta monitoring) as a ‘high-risk’ AI use case. This would bring procurement within the scope of the EU AI Act, but would still be largely ineffectual due to the exemptions in Article 6(3) EU AI Act. Therefore, the Public Procurement Act should also specify that Article 6(3) EU AI Act is inapplicable to the use of AI for procurement. This would go a long way in addressing the governance of AI use by public buyers.
As a complement—or a rather weak substitute, if modifying the EU AI Act in this way is not (politically) feasible—the Public Procurement Act could include an explicit provision on AI use for procurement to make public buyers directly responsible for the consequences of AI use for procurement, regardless of any responsibility or liability that can be sought from the providers of the used AI solutions—and this could be coupled by an obligation to develop, publish and constantly update a policy on the use of AI for procurement. This should include specific approaches to testing for bias and anticompetitive effects potentially arising from AI use (for discussion, see here).
Second, to mirror this, the use of AI by economic operators participating in procurement procedures would also need regulating. At a minimum, this should be reflected in the rules on exclusion (which need a significant overhaul more generally, in particular to make all exclusion grounds mandatory if the instrument is to remain as a Regulation) and an explicit exclusion ground should cover the misuse of AI in the participation in procurement procedures or in the execution of public contracts.
Finally, AI procurement needs to be given legislative coverage. As there is no chance that a full regime can be included in an Act, and much less that this would be sufficiently dynamic and adaptable, the Public Procurement Act should include a new article mandating the Commission and the Member States, jointly, to develop and constantly update a set of mandatory (not model) clauses for AI procurement. These would need to be a lot more detailed and implementable than the current clauses, and it seems to me that the future clauses need to be jurisdiction-specific (not merely translated into relevant languages) for them to adequately bridge the gap between public and private law rules (including liability) and jurisdictional variations. It would also be important to consider comparative approaches and to reflect on and learn from, eg recent developments in the US with the adoption by the General Services Administration (GSA) of a new clause for incorporation into government contracts of artificial intelligence systems (552.239-7001, Basic Safeguarding of Data within Large Language Model Artificial Intelligence Systems).
These interventions would still need refining and careful drafting and, most likely, would probably still be insufficient to fully assure AI procurement and the use of AI for procurement and its oversight, but they would provide a firm statutory grounding on which to continue the regulatory conversation.