‘EU Member States only have about one year to make crucial decisions that will affect the procurement data architecture of the EU and the likelihood of successful adoption of digital technologies for procurement governance for years or decades to come’. Put like that, the relevance of the approaching deadline for the national implementation of new procurement eForms may grab more attention than the alternative statement that ‘in just about a year, new eForms will be mandatory for publication of procurement notices in TED’.
This latter more technical (obscure, and uninspiring?) understanding of the new eForms seems to have been dominating the approach to eForms implementation, which does not seem to have generally gained a high profile in domestic policy-making at EU Member State level despite the Publications Office’s efforts.
In this post, I reflect about the strategic importance of the eForms implementation for the digitalisation of procurement, the limited incentives for an ambitious implementation that stem from the voluntary approach of the most innovative aspects of the new eForms, and the opportunity that would be lost with a minimalistic approach to compliance with the new rules. I argue that it is urgent for EU Member States to get serious about building a procurement data architecture that facilitates the uptake of digital technologies for procurement governance across the EU, which requires an ambitious implementation of eForms beyond their minimum mandatory requirements.
eForms: some background
The EU is in the process of reforming the exchange of information about procurement procedures. This information exchange is mandated by the EU procurement rules, which regulate a variety of procurement notices with the two-fold objective of (i) fostering cross-border competition for public contracts and (ii) facilitating the oversight of procurement practices by the Member States, both in relation to the specific procedure (eg to enable access to remedies) and from a broad policy perspective (eg through the Single Market Scoreboard). In other words, this information exchange underpins the EU’s approach to procurement transparency, which mainly translates into publication of notices in the Tenders Electronic Daily (TED).
A 2019 Implementing Regulation established new standard forms for the publication of notices in the field of public procurement (eForms). The Implementing Regulation is accompanied by a detailed Implementation Handbook. The transition to eForms is about to hit a crucial milestone with the authorisation for their voluntary use from 14 November 2022, in parallel with the continued use of current forms. Following that, eForms will be mandatory and the only accepted format for publication of TED notices from 25 October 2023. There will thus have been a very long implementation period (of over four years), including an also lengthy (11-month) experimentation period about to start. This contrasts with previous revisions of the TED templates, which had given under six months’ notice (eg in 2015) or even just a 20-day implementation period (eg in 2011). This extended implementation period is reflective of the fact that the transition of eForms is not merely a matter of replacing a set of forms with another.
Indeed, eForms are not solely the new templates for the collection of information to be published in TED. eForms represent the EU’s open standard for publishing public procurement data — or, in other words, the ‘EU OCDS’ (which goes much beyond the OCDS mapping of the current TED forms). The importance of the implementation of a new data standard has been highlighted at strategic level, as this is the cornerstone of the EU’s efforts to improve the availability and quality of procurement data, which remain suboptimal (to say the least) despite continued efforts to improve the quality and (re)usability of TED data.
In that regard, the 2020 European strategy for data, emphasised that ‘Public procurement data are essential to improve transparency and accountability of public spending, fighting corruption and improving spending quality. Public procurement data is spread over several systems in the Member States, made available in different formats and is not easily possible to use for policy purposes in real-time. In many cases, the data quality needs to be improved.’ The European Commission now stresses how ‘eForms are at the core of the digital transformation of public procurement in the EU. Through the use of a common standard and terminology, they can significantly improve the quality and analysis of data’ (emphasis added).
It should thus be clear that the eForms implementation is not only about low level form-filling, but also (or primarily) about building a procurement data architecture that facilitates the uptake of digital technologies for procurement governance across the EU. Therefore, the implementation of eForms and the related data standard seeks to achieve two goals: first, to ensure the data quality (eg standardisation, machine-readability) required to facilitate its automated treatment for the purposes of publication of procurement notices mandated by EU law (ie their primary use); and, second, to build a data architecture that can facilitate the accumulation of big data so that advanced data analytics can be deployed by re-users of procurement data. This second(ary) goal is particularly relevant to our discussion. This requires some unpacking.
The importance of data for the deployment of digital technologies
It is generally accepted that quality (big) data is the primary requirement for the deployment of digital technologies to extract data-driven insights, as well as to automate menial back-office tasks. In a detailed analysis of these technologies, I stress the relevance of procurement data across technological solutions that could be deployed to improve procurement governance. In short, the outcome of robotic process automation (RPA) can only be as good as its sources of information, and adequate machine learning (ML) solutions can only be trained on high-quality big data—which thus conditions the possibility of developing recommender systems, chatbots, or algorithmic screens for procurement monitoring and oversight. Distributed Ledger Technology (DLT) systems (aka blockchain) can manage data, but cannot verify its content, accuracy, or reliability. Internet of Things (IoT) applications and software oracles can automatically capture data, which can alleviate some of the difficulties in generating an adequate data infrastructure. But this is only in relation with the observation of the ‘real world’ or in relation to digitally available information, which quality raises the same issues as other sources of data. In short, all digital technologies are data-centric or, more clearly, data-dependent.
Given the crucial relevance of data across digital technologies, it is hard to emphasise how any shortcomings in the enabling data architecture curtail the likelihood of successful adoption of digital technologies for procurement governance. With inadequate data, it may simply be impossible to develop digital solutions at all. And the development and adoption of digital solutions developed on poor or inadequate data can generate further problems—eg skewing decision-making on the basis of inadequately derived ‘data insights’. Ultimately, then, ensuring that adequate data is available to develop digital governance solutions is a challenging but unavoidable requirement in the process of procurement digitalisation. Success, or lack of it, in the creation of an enabling data architecture will determine the viability of the deployment of digital technologies more generally. From this perspective, the implementation of eForms gains clear strategic importance.
eForms Implementation: a flexible model
Implementing eForms is not an easy task. The migration towards eForms requires a complete redesign of information exchange mechanisms. eForms are designed around universal business language and involve the use of a much more structured information schema, compatible with the EU’s eProcurement Ontology, than the current TED forms. eForms are also meant to collect a larger amount of information than current TED forms, especially in relation to sub-units within a tender, such as lots, or in relation to framework agreements. eForms are meant to be flexible and regularly revised, in particular to add new fields to facilitate data capture in relation to specific EU-mandated requirements in procurement, such as in relation with the clean vehicles rules (with some changes already coming up, likely in November 2022).
From an informational point of view, the main constraint that remains despite the adoption of eForms is that their mandatory content is determined by existing obligations to report and publish tender-specific information under the current EU procurement rules, as well as to meet broader reporting requirements under international and EU law (eg the WTO GPA). This mandatory content is thus rather limited. Ultimately, eForms’ main concentration is on disseminating details of contract opportunities and capturing different aspects of decision-making by the contracting authorities. Given the process-orientedness and transactional focus of the procurement rules, most of the information to be mandatorily captured by the eForms concerns the scope and design of the tender procedure, some aspects concerning the award and formal implementation of the contract, as well as some minimal data points concerning its material outcome—primarily limited to the winning tender. As the Director-General of the Publications Office put it an eForms workshop yesterday, the new eForms will provide information on ‘who buys what, from whom and for what price’. While some of that information (especially in relation to the winning tender) will be reflective of broader market conditions, and while the accumulation of information across procurement procedures can progressively generate a broader view of (some of) the relevant markets, it is worth stressing that eForms are not designed as a tool of market intelligence.
Indeed, eForms do not capture the entirety of information generated by a procurement process and, as mentioned, their mandatory content is rather limited. eForms do include several voluntary or optional fields, and they could be adapted for some voluntary uses, such as in relation to detection of collusion in procurement, or in relation to the beneficial ownership of tenderers and subcontractors. Extensive use of voluntary fields and the development of additional fields and uses could contribute to generating data that enabled the deployment of digital technologies for the purposes of eg market intelligence, integrity checks, or other sorts of (policy-related) analysis. For example, there are voluntary fields in relation to green, social or innovation procurement, which could serve as the basis for data-driven insights into how to maximise the effects of such policy interventions. There are also voluntary fields concerning procurement challenges and disputes, which could facilitate a monitoring of eg areas requiring guidance or training. However, while the eForms are flexible, include voluntary fields, and the schema facilitates the development of additional fields, is it unclear that adequate incentives exist for adoption beyond their mandatory minimum content.
Implementation in two tiers
The fact that eForms are in part mandatory and in part voluntary will most likely result in two separate tiers of eForms implementation across the EU. Tier 1 will solely concern the collection and exchange of information mandated by EU law, that is the minimum mandatory eForm content. Tier 2 will concern the optional collection and exchange of a much larger volume of information concerning eg the entirety of tenders received, as well as qualitative information on eg specific policy goals embedded in a tender process. Of course, in the absence of coordination, a (large) degree of variation within Tier 2 can be expected. Tier 2 is potentially very important for (digital) procurement governance, but there is no guarantee that Member States will decide to implement eForms covering it.
One of the major obstacles to the broad adoption of a procurement data model so far, at least in the European Union, relates to the slow uptake of e-procurement (as discussed eg here). Without an underlying highly automated e-procurement system, the generation and capture of procurement data is a main challenge, as it is a labour-intensive process prone to input error. The entry into force of the eForms rules could serve as a further push for the completion of the transition to e-procurement—at least in relation to procurement covered by EU law (as below thresholds procurement is a voluntary potential use of eForms). However, it is also possible that low e-procurement uptake and generalised unsophisticated approaches to e-procurement (eg reduced automation) will limit the future functionality of eForms, with Member States that have so far lagged behind restricting the use of eForms to tier 1. Non life-cycle (automated) e-procurement systems may require manual inputs into the new eForms (or the databases from which they can draw information) and this implies that there is a direct cost to the implementation of each additional (voluntary) data field. Contracting authorities may not perceive the (potential) advantages of incurring those costs, or may more simply be constrained by their available budget. A collective action problem arises here, as the cost of adding more data to the eForms is to be shouldered by each public buyer, while the ensuing big data would potentially benefit everyone (especially as it will be published—although there are also possibilities to capture but not publish information that should be explored, at least to prevent excessive market transparency; but let’s park that issue for now) and perhaps in particular data re-users offering for pay added-value services.
In direct relation to this, and compounding the (dis)incentives problem, the possibility (or likelihood) of minimal implementation is compounded by the fact that, in many Member States, the operational adaptation to eForms does not directly concern public sector entities, but rather their service providers. e-procurement services providers compete for the provision of large volume, entirely standardised platform services, which are markets characterised by small operational margins. This creates incentives for a minimal adaptation of current e-sending systems and disincentives for the inclusion of added-value (data) services potentially unlikely to be used by public buyers. Some (or most) optional aspects of the eForm implementation will thus remain unused due to these market structure and dynamics, which does not clearly incentivise a race to the top (unless there is clear demand pull for it).
With some more nuance, it should be stressed that it is also possible that the adoption of eForms is uneven within a given jurisdiction where the voluntary character of parts of the eForm is kept (rather than made mandatory across the board through domestic legislation), with advanced procurement entities (eg central purchasing bodies, or large buyers) adopting tier 2 eForms, and (most) other public buyers limiting themselves to tier 1.
Ensuing data fragmentation
While this variety of approaches across the EU and within a Member State would not pose legal challenges, it would have a major effect on the utility of the eForms-generated data for the purposes of eg developing ML solutions, as the data would be fragmented, hardly representative of important aspects of procurement (markets), and could hardly be generalisable. The only consistent data would be that covered by tier 1 (ie mandatory and standardised implementation) and this would limit the potential use cases for the deployment of digital technologies—with some possibly limited to the procurement remit of the specific institutions with tier 2 implementations.
Relatedly, it should be stressed that, despite the effort to harmonise the underlying data architecture and link it to the Procurement Ontology, the Implementation Handbook makes clear that ‘eForms are not an “off the shelf” product that can be implemented only by IT developers. Instead, before developers start working, procurement policy decision-makers have to make a wide range of policy decisions on how eForms should be implemented’ in the different Member States.
This poses an additional challenge from the perspective of data quality (and consistency), as there are many fields to be tailored in the eForms implementation process that can result in significant discrepancies in the underlying understanding or methodology to determine them, in addition to the risk of potential further divergence stemming from the domestic interpretation of very similar requirements. This simply extends to the digital data world the current situation, eg in relation to diverging understandings of what is ‘recyclable’ or what is ‘social value’ and how to measure them. Whenever open-ended concepts are used, the data may be a poor source for comparative and aggregate analysis. Where there are other sources of standardisation or methodology, this issue may be minimised—eg in relation to the green public procurement criteria developed in the EU, if they are properly used. However, where there are no outside or additional sources of harmonisation, it seems that there is scope for quite a few difficult issues in trying to develop digital solutions on top of eForms data, except in relation to quantitative issues or in relation to information structured in clearly defined categories—which will mainly link back to the design of the procurement.
An opportunity about to be lost?
Overall, while the implementation of eForms could in theory build a big data architecture and facilitate the development of ML solutions, there are many challenges ahead and the generalised adoption of tier 2 eForms implementations seems unlikely, unless Member States make a positive decision in the process of national adoption. The importance of an ambitious tier 2 implementation of eForms should be assessed in light of its downstream importance for the potential deployment of digital technologies to extract data-driven insights and to automate parts of the procurement process. A minimalistic implementation of eForms would significantly constrain future possibilities of procurement digitalisation. Primarily in the specific jurisdiction, but also with spillover effects across the EU.
Therefore, a minimalistic eForms implementation approach would perpetuate (most of the) data deficit that prevents effective procurement digitalisation. It would be a short-sighted saving. Moreover, the effects of a ‘middle of the road’ approach should also be considered. A minimalistic implementation with a view to a more ambitious extension down the line could have short-term gains, but would delay the possibility of deploying digital technologies because the gains resulting from the data architecture are not immediate. In most cases, it will be necessary to wait for the accumulation of sufficiently big data. In some cases of infrequent procurement, missing data points will generate further time lags in the extraction of valuable insights. It is no exaggeration that every data point not captured carries an opportunity cost.
If Member States are serious about the digitalisation of public procurement, they will make the most of the coming year to develop tier 2 eForms implementations in their jurisdiction. They should also keep an eye on cross-border coordination. And the European Commission, both DG GROW and the Publications Office, would do well to put as much pressure on Member States as possible.