In its Judgment of 14 July 2016 in TNS Dimarso, C-6/15, EU:C:2016:555, the Court of Justice of the European Union (CJEU) issued some important clarifications on the requirements applicable to the disclosure of evaluation methods under the EU public procurement rules. However, it also turned down the opportunity of clarifying what are the limits of the discretion that contracting authorities enjoy when deciding which evaluation methods to use and, more importantly, it failed to address the important and quite specific concerns about the use of 'soft quality metrics' that AG Mengozzi had raised in his Opinion in this case (as discussed here, where background to the case is offered).
The case broadly raised two main legal issues. First, whether in addition to the disclosure of the award criteria and their weighting (as required by Art 53(2) Dir 2004/18 and now Art 67(5) Dir 2014/24), contracting authorities must also disclose in the tender documentation, or at some point prior to the review of the offers, the evaluation methods they plan to use in the assessment of the tenders. Second, whether having disclosed a numerical weighting applicable to the quality and price criteria (50/100 each), the contracting authority was right to assess the quality criterion in accordance with a soft qualitative ‘high — satisfactory — low’ scale, not referred to in the contract documents.
no obligation to disclose (or indeed establish) evaluation rules prior to the review of the tenders
Regarding the first issue, after reiterating its case law on the purpose of the rules on disclosure of award criteria and their weighting, and stressing their relevance in ensuring equality of treatment between tenderers both when they formulate their tenders and when those tenders are being assessed by the contracting authority (para 22), the CJEU adopted a position that I find surprising. The CJEU stressed that
it is possible for a contracting authority to determine, after expiry of the time limit for submitting tenders, weighting factors for the sub-criteria which correspond in essence to the criteria previously brought to the tenderers’ attention, provided that three conditions are met, namely that that subsequent determination, first, does not alter the criteria for the award of the contract set out in the tender specifications or contract notice; secondly, does not contain elements which, if they had been known at the time the tenders were prepared, could have affected their preparation; and, thirdly, was not adopted on the basis of matters likely to give rise to discrimination against one of the tenderers (see judgment of 21 July 2011 in Evropaïki Dynamiki v EMSA, C‑252/10 P, not published, EU:C:2011:512, paragraph 33 and the case-law cited) (C-6/15, para 26).
However, it did not apply this reasoning by analogy to evaluation methods as could have been expected. On the contrary, the CJEU adopted a very lenient approach and, after confirming that neither the rules in the Directive nor the previous case law referred to an obligation to disclose evaluation methods, it went on to establish that
29 ... an evaluation committee must be able to have some leeway in carrying out its task and, thus, it may, without amending the contract award criteria set out in the tender specifications or the contract notice, structure its own work of examining and analysing the submitted tenders (see judgment of 21 July 2011 in Evropaïki Dynamiki v EMSA, C‑252/10 P, not published, EU:C:2011:512, paragraph 35).
30 That leeway is also justified by practical considerations. The contracting authority must be able to adapt the method of evaluation that it will apply in order to assess and rank the tenders in accordance with the circumstances of the case.
31 In accordance with the principles governing the award of contracts provided for in Article 2 of Directive 2004/18 and in order to avoid any risk of favouritism, the method of evaluation applied by the contracting authority in order to specifically evaluate and rank the tenders cannot, in principle, be determined after the opening of the tenders by the contracting authority. However, in the event that the determination of that method is not possible for demonstrable reasons before the opening of the tenders, as noted by the Belgian Government, the contracting authority cannot be criticised for having established it only after that authority, or its evaluation committee, reviewed the content of the tenders.
32 In any event, pursuant to the principles governing the award of contracts ... the determination by the contracting authority of the method of evaluation after the publication of the contract notice or the tender specifications cannot have the effect of altering the award criteria or their relative weighting (C-6/15, paras 29-32, emphasis added).
The reasoning of the CJEU raises two surprising issues, in my view. First, the CJEU seems to conflate the need for the contracting authority to establish an evaluation method that is adapted to the particularities of a given tender (which seems correct, para 30) with the need for the contracting authority to be able to do that at any time (which seems incorrect, para 31). Accepting that the contracting authority can design ad hoc evaluation methods for each of the contracts it tenders does not imply that it can leave this important aspect of the evaluation process for a late stage. Logically, it would seem that setting the award criteria, their weighting and establishing the rules according to which they will be evaluated are different aspects of one same decision: how will the tenders be evaluated so that the contracting authority can decide which one is the most economically advantageous?
It does not seem diligent for the contracting authority to set out the award criteria and their weighting without having determined the way these will be applied in the evaluation. It also seems to create unnecessary uncertainty to tenderers. This is very clear in relation to the use of automatic formulae in electronic auctions, which need to be disclosed to the tenderers prior to their use (Art 54(5) Dir 2004/18 and Art 35(6) Dir 2014/24). There does not seem to be a good reason for these considerations not to apply to the use of evaluation methods and to require that the contracting authority is diligent in setting them up in a timely manner (ie when it is setting out the award criteria and their weighting).
Second, and more surprisingly, the CJEU fails to extend to the evaluation method the most obvious and minimal guarantee to avoid (impossible to prove) discrimination, ie determining the illegality of establishing (evaluation) criteria relevant for the assessment of the tenders after the evaluation committee has reviewed them (para 31). Before anything else, it must be noted that the CJEU accepts that 'the method of evaluation applied by the contracting authority in order to specifically evaluate and rank the tenders cannot, in principle, be determined after the opening of the tenders by the contracting authority'. The reasoning should not have been as a matter of principle, but as a point of absolute requirement.
However, it is not clear why the CJEU concedes that 'in the event that the determination of that method is not possible for demonstrable reasons before the opening of the tenders, as noted by the Belgian Government, the contracting authority cannot be criticised for having established it only after that authority, or its evaluation committee, reviewed the content of the tenders'. There is no indication whatsoever in the Judgment of which reasons may have been adduced by the Belgian Government to try to justify the impossibility of establishing the evaluation method before having reviewed the tenders. This is amazing because it makes it impossible to understand where the threshold of impossibility lies and, more importantly, because there do not seem to be any good reasons to accept that a diligent evaluation committee can be allowed to decide on the evaluation method after it has already seen the content of the tenders. Whether this is done in a presential meeting or remotely, there is no justification for the assessors not to agree on the evaluation rules first (and document them), and then proceed to the evaluation. In my view, the CJEU has neglected the need to ensure the right to good administration and, in particular, the need to ensure the most basic guarantees that tenderers are treated impartially and fairly, and that relevant matters are dealt with in a timely fashion (as required by Article 41(1) Charted of Fundamental Rights).
The final protection that the CJEU tries to (re)establish in the case by stressing that 'the determination by the contracting authority of the method of evaluation after the publication of the contract notice or the tender specifications cannot have the effect of altering the award criteria or their relative weighting' is inane and insufficient because the possibility of establishing and playing with evaluation rules after having seen the content of the tenders leaves way too much scope to coming up with rules that allow for an ex post rationalisation of the choice of a given winning in tender without necessarily violating the pre-disclosed information on the applicable award criteria and weightings. This deserves stern criticism.
the use of 'soft quality metrics' in the evaluation of tenders
Moving on to the second issue concerning the use of 'soft quality metrics', such as the ‘high — satisfactory — low’ scale in the case at issue, in my opinion, the CJEU also carried out a defective analysis. The shortcomings of the analysis derive from the fact that the CJEU uses the answer to this second aspect to try to compensate for the weakness of its answer to the first question. Indeed, the CJEU premises the analysis of the use of the 'soft quality metrics' on the assessment of whether their use altered the applicable award criteria and their relative weighting. The reasoning of the CJEU is as follows:
35 It appears that that procedure did not make it possible to reflect, when ranking the tenderers in order to identify the most economically advantageous tender, differences in the quality of their tenders relative to their price, while taking account of the relative weighting of the award criteria resulting from the indication ‘(50/100)’. In particular, it appears that that procedure was capable of affecting the price criterion by giving it decisive weight relative to the tenders ranked in the [‘high — satisfactory — low’] scale of quality ... It is for the referring court to ascertain whether the relative weighting of each of the award criteria published in the contract notice was in fact complied with by the contracting authority during the evaluation of the tenders.
36 While the contracting authority may use a scale for the evaluation of one of the award criteria without it being published in the call for tenders or the tender specifications, that scale may not, however ... have the effect of altering the relative weighting of the award criteria published in those documents (C-6/15, paras 35-36, emphasis added).
I find this problematic because it does not address the core problem of using 'soft quality metrics' at all. Should they have been disclosed to tenderers or, more importantly, should they have been published in the tender documentation together with an explanation of why specific weightings could not be established, the use of this scale would be unobjectionable under the TNS Dimarso test, while still allowing for very subjective and difficult to objectively verify comparisons of the tenders. This leaves the question of which evaluation rules are compatible with the two main requirements in the evaluation of tenders--ie that the award rules, not only the award criteria themselves and their weighting, have to '(i) to be linked to the subject matter of the contract (ie, to be ‘relevant’), and (ii) to allow the contracting authority actually to determine which tender is economically the most advantageous (ie, to be ‘enabling’)' [A Sanchez-Graells, Public Procurement and the EU Competition Rules, 2nd edn (Oxford, Hart, 2015) 380]. By failing to clarify that 'soft quality metrics' are not enabling and do not provide sufficient objectivity to the evaluation process so as to make sure that the contracting authority does not overstep the limits of its discretion, in my view, the CJEU has left too much space for manipulation in the assessment of tenders.
This is something I had criticised [A Sanchez-Graells, Public Procurement and the EU Competition Rules, 2nd edn (Oxford, Hart, 2015) 388], even if in relation to the award criteria (but the arguments apply equally to evaluation rules meant to assess them] considering the following:
Restrictions Derived from the Inclusion of Non-Quantifiable or Subjective Award Criteria, and the Ensuing Need to Objectify Treatment of Qualitative Criteria. Another way in which the selection and weighting of award criteria could give rise to distortions of competition—and, probably, to discrimination amongst tenderers—would be through the introduction of non-quantifiable criteria, or essentially qualitative or subjective criteria that significantly diminished the possibilities of an overall objective appraisal of the tenders or conferred on contracting authorities unrestricted freedom of choice amongst tenderers. In this regard, even if article 67(2)(a) of Directive 2014/24 allows for the taking into consideration of this type of criterion—referring, in general terms, to criteria such as ‘technical merit’ or ‘aesthetic characteristics’—the requirements of relevance and enabling character of the award criteria (see above, this section), as well as the need to avoid conferring on contracting authorities unrestricted freedom of choice and to ensure that the award criteria make provision for an objective assessment of tenderers, should be taken into particular consideration and constrain the decisions adopted by the public buyer.[1]
As regards the requirement of relevance of such qualitative award criteria, it should be stressed that the circumstances under which considerations such as aesthetic characteristics or technical merit will be relevant and material to the subject-matter of the contract are relatively limited (at least if they are unrelated to performance or functional requirements, which are quantifiable and, hence, do not generate significant difficulties). Moreover, it is submitted that they will generally be associated with tenders that should be ruled by the requirements applicable to design contests—which are specifically regulated and set special rules in this respect (see arts 78 to 82 of dir 2014/24),[2] particularly aimed at ensuring the objectivity and independence of the members of the committee entrusted with the evaluation of qualitative or subjective elements of the proposals. Consequently, aesthetic characteristics or technical merit might be assigned very limited relevance in other types of tendering procedures. The substantial irrelevance of such qualitative or non-quantifiable aspects will, then, require only limited consideration in the majority of the cases, if at all.
Moreover, in order to ensure transparency and impartiality, contracting authorities should (as far as possible) set objective or quantifiable proxies to measure primarily subjective or qualitative characteristics of the tenders; or, at least, set up mechanisms (possibly based on the rules regarding design contests) to ensure an impartial appraisal of subjective or qualitative dimensions of the tenders. If such quantification, or ‘proximisation’ or approximation, is possible, the possibilities for discrimination or distortion of competition will be smaller. Consequently, the adoption of this requirement seems desirable whenever its implementation is feasible.
Therefore, a restrictive approach towards the permissibility of the use of these criteria as the basis for the award of contracts—again, in cases other than design contests—seems appropriate. Consequently, this type of consideration should remain as a secondary criterion, or as a rather marginal complement, to objective and easily quantifiable criteria used to determine the award of the contract to the most economically advantageous tender. Along these lines, and attending to the subject-matter of the contract, contracting authorities should give proper weighting to qualitative or subjective criteria (even if ‘quantified’)—which, in our opinion, should be rather limited and marginal in most instances.
To sum up, it is submitted that contracting authorities are bound to ensure the objective and transparent assessment of tenders, particularly by i) avoiding undue recourse to qualitative or non-quantifiable (subjective) award criteria in procedures other than design contests, and ii) assigning them a proper (limited) weighting; and, in general, they are under a duty to exercise self-restraint in their decisions regarding such criteria, particularly when failure to do so could result in their exercise of unrestricted freedom of choice amongst tenderers and/or generate distortions of competition or discrimination of tenderers.
[1] See: S Arrowsmith, The Law of Public and Utilities Procurement. Regulation in the EU and the UK, Vol. 1, 3rd edn (London, Sweet & Maxwell, 2014) 766–71.
[2] See S Arrowsmith, The Law of Public and Utilities Procurement, 2nd edn (London, Sweet & Maxwell, 2005) 829–39; PA Trepte, Public Procurement in the EU: A Practitioner’s Guide, 2nd edn (Oxford, Oxford University Press, 2007) 232–4; and C Bovis, EC Public Procurement: Case Law and Regulation (Oxford, Oxford University Press, 2006) 248–51.