July 01, 2019 Public Contract Law Journal

Feedback and Supplier Debriefing: The Unloved Child of SME-Friendly Procurement Pratice

by Ceri Evans & Dermot Cahill

Ceri Evans is Lecturer in Procurement & Innovation and Senior Procurement Specialist at the Institute for Competition & Procurement Studies at Bangor University.

Professor Dermot Cahill is Chairman of the Institute for Competition &  Procurement Studies and Head of Bangor University Law School.

I.  Introduction1

Initiatives to embrace small and medium-sized enterprises (SMEs)2 in public sector procurement have been widespread in recent years across the globe, and yet SMEs still remain significantly under-represented in public sector procurement in comparison to their economic contribution and their number.3

One area of SME-friendly procurement that arguably has not been given the attention it deserves by major institutions, such as the European Com- mission, is the provision of public procurer feedback to SMEs, or rather the provision of meaningful feedback to SMEs following their submission of an unsuccessful public sector tender. Academic research into this topic has also been limited by the fact that, up until now, there have been no empirical studies into public sector bodies’ feedback practices to SMEs.4 Feedback is, however, of critical importance to SMEs in helping them understand why they have failed to win in a public sector competition and what is required for future success.5

In drawing on evidence gathered from feedback letters analyzed as part of a European Union (EU)-funded procurement research project,6 this article examines actual feedback practice for the first time. It does so in the form of a content-based analysis of feedback issued to disappointed bidders by eighteen separate United Kingdom (UK) public sector bodies (including central government and local government organisations, the police force, and universities) across thirty-nine different procurement exercises.7 It provides empirical evidence regarding the extent of feedback provision and the qualitative nature of the feedback provided, that bridges the knowledge gap between SME claims regarding the feedback they have received and procurer assertions about their feedback practice.8

The analysis enhances our understanding of whether feedback practices meet the needs of SMEs, and in so doing, will explore whether there is a disconnect between policy and practice in relation to improving SME access to the public sector marketplace, specifically in the context of non-transparent public procurer feedback provision to SMEs following unsuccessful bid submissions. The findings from this study have ramifications not just for public procurement practice in the UK and EU, but also for other jurisdictions throughout the world that have adopted a proactive stance towards embracing SMEs in public sector procurement and removing barriers to their participation.

This article also proposes the first-ever system for categorizing different components of feedback, as well as a unique Universal Feedback Methodology that public sector bodies can use to provide “good” feedback to SMEs in a standardized fashion regardless of contract size, complexity, or jurisdiction. Public procurers who adopt this methodology will be confident they are operating in an SME-friendly fashion, satisfying transparency and integrity requirements, while at the same time being legally compliant with national and EU procurement case law and legislation and enabling SMEs to defend their position if procedural unfairness has occurred. This should prove useful to academics and practitioners alike. It will also give assurance to disappointed SMEs that the feedback they receive will allow them to be sufficiently informed so that they can submit a stronger bid next time around, while also giving them confidence that this feedback has met a standard of fairness that complies with their legal entitlements in enabling them to understand why they were not selected as the winner.

The rest of this article will proceed as follows: Part II provides contextual background related to SMEs and public procurement, discussing the significance of feedback from both sides of the supplier-purchaser interface. Part III provides a review of existing literature, addressing the anecdotal nature of literature on the subject and the issue of feedback and public procurement policy. In Part IV, the article provides the legal analysis underpinning the study, with Part V detailing the research methodology. Part VI presents and discusses the research findings, with Part VII discussing the implications for policy and practice, including the proposed Universal Feedback Methodology. The final Part provides concluding remarks.

II.  SMES and Feedback: Significance and Content

A.  SME Access to Public Sector Procurement

Many countries throughout the world have seemingly adopted an aggressive stance towards embracing SMEs in public procurement in recent years, as part of their procurement modernization efforts. For example, according to the Organisation for Economic Co-operation and Development (OECD), eighty-five percent of their member countries have instituted measures9 to help “level the playing field” for SMEs in public sector procurement.10 At an EU level, the 2014 EU Public Procurement Directives11 include measures to facilitate SME access to public procurement.12 In October 2017, the European Commission set out six policy priorities for public procurement and made public procurement more accessible for companies, in particular SMEs, one of these priority improvement areas.13 This commitment by the EU to improving access to public sector contracts for SMEs is not new, but nontheless serves to remind us that this agenda remains firmly on the Commission’s table. The UK Government has set a target of thirty-three percent of central government procurement spend (either directly or via the supply chain) going to SMEs by 2022,14 and it has implemented a number of SME-friendly initiatives15 with the aim of increasing spend with SMEs and removing barriers to their participation.

Statistics relating to EU public sector contracts won by SMEs differ somewhat depending on the data source. For below EU threshold contracts,16 SMEs secure seventy-three percent of the volume of contracts, whereas for above EU threshold contracts, the equivalent figure is fifty-six percent.17 Latest statistics from the European Commission show that SMEs at present only win around forty-five percent of the total contract value of all above EU threshold contracts.18 However one analyzes it, the fact remains that SMEs are under-represented in public procurement in comparison to their contribution to the economy (they produce more than fifty percent of GDP) and their numbers (they account for ninety-nine point eight percent of all EU enterprises).19 When one considers that SMEs created eighty-five percent of all new jobs in Europe between 2002 and 2010,20 the gap between their contribution to the economy and under-representation in terms of winning public sector contracts becomes even more acute.

Interestingly, at a national level in the UK, there is some evidence to suggest that the proportion of contracts won by SMEs is actually in decline despite the proactive, SME-friendly stance the UK has adopted. For example, the propotion of SMEs who worked for the public sector in 2016 was twenty-three percent, in comparison to twenty-five percent in 2014,21 (although a more longitudinal comparative analysis over a number of years is required to draw concrete conclusions).

As part of this wider discussion, it is also important not to forget to mention the widely recognized symbiotic relationship between SMEs and public procurement. This mutually beneficial relationship has been well documented in the literature: for SME suppliers, the benefits include the promise of longstanding, stable customers who pay on time, a potential income buffer during recessionary periods, and the potential to gain esteemed references.22 SME involvement in public sector contracting also has a compelling appeal for public sector organisations because SMEs are innovative, flexible, and possess the capacity to provide niche offerings.23 The challenge over the years for policymakers has been to promote SME involvement in the public sector marketplace (worth €2 trillion per year in the EU alone and representing fourteen percent of EU Member States’ GDP)24 without providing them with any undue advantage.

B.  Benefits to Feedback

While there is no universally-accepted definition of supplier feedback in the literature, nearly all references to feedback or “debriefing” (as it is also known) refer to a number of common underpinning elements.25 These include: thinking of feedback as an essential element of good procurement practice; as a mechanism to help suppliers improve their performance; as a means for procurers to ensure continuous improvement; and as a means to explain the basis of a public body’s decision to award a contract to one bidder over another, or to explain why a supplier has not reached the award stage of a tender competition.26

Indeed, feedback has the potential to benefit both parties of the supplier- purchaser interface. For the supplier, it can help them reassess and understand changes they might need to make to their business and/ or their tendering approach in order to make them tender-ready or contract-ready and hence make future bids more successful. It also serves to enlighten them as to how public procurement procedures work, and importantly, how they differ from the private sector.27 Of equal significance is the fact that it can provide them with the reassurance and confidence that the correct process was conducted with integrity and in a transparent fashion.28

From the purchaser perspective, the benefits are likewise potentially numerous. Feedback can encourage improved future tender submissions, enhancing the scope for public bodies to contract with the best value-for-money supplier in the marketplace and not just those that gain an edge because they can afford to employ specialist tender-writers.29 Significantly, providing feedback creates an opportunity for public sector organisations to demonstrate that they operate in a fair, open, honest, and ethical way, thereby potentially changing perceptions and enhancing the prospect of suppliers viewing them as attractive customers (i.e., encouraging suppliers to submit future bids rather than suppliers becoming disillusioned and exiting the marketplace).30 Additionally, feedback provides further opportunity to educate suppliers that cost is not the only concern of purchasers and that the wider pursuit of value is often as  – or sometimes even more – important.31

Beyond these potential benefits for buyers, it is also imperative to acknowledge that the EU Public Procurement Directives entitle disappointed bidders to receive explanations for award decisions that are sufficient to allow them to understand the decision.32 There is also, of course, a “moral” argument that procurers should provide useful feedback because of the costs suppliers incur in the preparation of their bids and the fact that taxpayers money is used to fund the public sector. Indeed, the cost for suppliers in bidding for public sector contracts has been estimated as £3,200 in the EU and £5,800 in the UK for a standard service contract.33 That does not even factor in the opportunity costs of this time. In comparison, it has been suggested that the cost of bidding for contracts in the private sector is typically ten percent to fifty percent less.34 If SME success rates in public sector tendering (widely heralded as typically being relatively low) are brought into the equation,35 then the importance of meaningful feedback from a moral perspective becomes even more stark.

C.  Drawbacks and Impediments to Feedback

Despite the numerous potential benefits of feedback, it is important to draw attention to the drawbacks to the provision of effective feedback – perceived or otherwise. For example, procurement staff will inevitably have concerns that legal proceedings could be invoked against a public body should a supplier contest some of the narrative provided within a feedback letter, particularly where comments might be interpreted to be subjective in nature.36 This may provide procurers with an unconscious or natural bias to limit the feedback information they provide to suppliers. The counterargument to this, as pointed out in historic UK Office of Government (OGC) supplier debriefing guidance, is that if the right decision was made in the first place and was well documented, then a procurer providing feedback should, by default, be able to explain or defend it with confidence.37

In relation to the above debate, public procurers should be cognizant of the fact that given the costs SMEs incur in the preparation of their bids (see supra Part II(B)), SMEs are likely to feel entitled to feedback that is meaningful in terms of explaining the strengths and weaknesses of their effort and confirms that a fair selection has occurred. Robust communication in this way (i.e., where the SME understands why they lost, and another company won), therefore potentially reduces litigation. In contrast, where public entities limit the feedback they provide, often in a misguided attempt to limit litigation, such feedback can appear disrespectful of the time and effort SMEs have put into preparing their bids and worse still, can give the impression the public body has something to hide. In the latter scenario, it can potentially put SME bidders in a position where they are forced to litigate in order to get full dis- closure as to why they lost.

Other impediments to effective feedback provision are less unique to feedback and instead represent generic barriers that arguably are common to a number of procurement activity areas. These barriers include time and resource constraints and the potential issue of procurers not being adequately trained and qualified; these aspects have been well researched in the literature.38

III.  Limitations of Existing Studies

A.    Feedback and Public Procurement Policy

The following section examines two illustrative examples of how feedback has been integrated into public procurement policy within EU Member States. It demonstrates that a relatively “loose” approach is being adopted in terms of what feedback should achieve, thereby giving public procurers considerable flexibility both in terms of the extent of feedback they provide and the qualitative nature of such feedback. This section also analyses EU level policy on this topic.

i.  Wales

Wales has adopted the Wales Procurement Policy Statement.39 Principle 8 of this Statement, entitled “Supplier Engagement and Innovation” advocates improving dialogue with suppliers to “help get the best response from the market place, to inform and educate suppliers, and to deliver optimum value for money.” Principle 8’s narrative refers to “ensuring debriefing provides adequate tender feedback” as a means for the Welsh public sector to achieve this objective.40

It is the use of the word “adequate” that fails to inspire confidence, and arguably it is not entirely consistent with the use of the word “educate” within the policy Principle itself.

ii.  Ireland

In Circular 10/14, the Irish government policy on facilitating SME involvement in public procurement is equally “loose.” It states:

Feedback[:] In order to prepare for future bids, it is very helpful for an unsuccessful tenderer to see which aspects of its bid were considered strong by a buyer and which aspects were considered weak. For contracts above EU thresholds for which advertising of contracts in the Official Journal of the EU is obligatory, buyers are required to give appropriate feedback to companies who have participated in a public procurement competition. For all other contracts buyers are strongly encouraged to provide written feedback as a matter of good practice.41

In dissecting this statement, there is a natural inclination to question the Irish government’s intentions and its impact on practice. For example, the use of the words “it is very helpful,” “appropriate feedback,” and “strongly encouraged to” could all be replaced with stronger adjectives such as “imperative,” “meaningful and transparent,” and “must.” It is beyond the scope of this article to thoroughly examine the reasons why a more hardline approach is not being adopted, but this example does serve to raise some serious question marks about the intentions of the government.

iii.  Latest EU Policy: A Missed Opportunity?

At this juncture, it is perhaps an opportune moment to mention that in the Commission’s latest procurement publication: Making Public Procurement work in and for Europe, the policy strand Improving access to procurement markets states that: “[a]nother way to improve access is to build trust among market players, including SMEs, to participate in procurement procedures.”42 While the report subsequently emphasizes the importance of the opportunity for suppliers to have an independent review of decisions,43 it is noticeable by its absence that there is no explicit mention of how the provision of meaningful44 and transparent feedback/ debriefing to suppliers can actually enhance trust amongst market players. This is a most surprising exclusion, and again, one has to question why this omission has been made. One possible explanation for this is that the EU Public Procurement Directives feedback/ debriefing requirements (see infra Part IV below) arguably focus on the need for contracting authorities to explain the basis of their decision to award a contract to one bidder over another, rather than emphasising how the provision of meaningful feedback can help suppliers improve. The question that must be raised, then, is whether legal compliance is taking precedence over the needs of SMEs in relation to feedback provision by public procurers.

B.  SME and Procurer Perceptions of Feedback Provision

A review of the literature provides a plethora of anecdotal support (i.e., without providing “hard evidence”) for the view that a lack of effective feedback provision is widespread and is proving an impediment to supplier engagement in public sector procurement, in particular SME engagement on a local, national, EU, and global level.

For example, in Wales (the authors’ home jurisdiction), the Barriers to Procurement Opportunity Report in 2009,45 in response to SME assertions that public bodies were often not providing meaningful feedback, made reference to sub-optimal feedback provision.46 At a UK level, similar findings were evident within the Glover report,47 with SMEs lamenting the poor feedback they were receiving.48 A more recent report by the UK Federation of Small Businesses (some ten years on from the Glover report) asserts that a lack of quality feedback provision to suppliers is still a major problem for SMEs according to their members. 49 Indeed, this report highlights the issue with feedback as being one of the five most serious barriers preventing small businesses from supplying to the UK public sector, and calls for the government to publish an action plan showing how they will enforce the provision of more detailed feedback by local government bodies.50 However, in a similar vein to the other studies mentioned, this report does not provide any direct evidence of this problem and instead has relied upon findings that have arisen from conducting interviews with small businesses and from canvassing the views of their members.51

Analysis of the UK Government’s Mystery Shopper Service (a service sitting within the Crown Commercial Service),52 which provides a route for suppliers to raise concerns about public procurement practice within English contracting authorities and to ensure public procurements do not impose unnecessary barriers to small businesses when bidding for public sector contracts, demonstrates that a number of cases concerning feedback have come to light since the Service’s launch in February 2011. While the number of cases is not high, those that have arisen highlight that provision of meaningful feedback to suppliers is not a given. For example, one recent issue raised by a supplier53 concerned the fact that they received very little information from the UK Cabinet Office Government Communication Service about why they scored lower than other bids for a low value tender exercise. Mystery Shopper investigated this case and upheld that very limited information was provided as to the relative advantages and disadvantages of the supplier`s unsuccessful bid, and it proposed recommendations54 for the Cabinet Office to improve their practice.

At an EU level, a similarly unflattering picture emerges. An evaluation of SMEs’ access to public procurement in the EU55 reported that “no debriefing” was one of the top four biggest problems companies face when participating in public procurement; but the evaluation does not provide any further insight into actual feedback practice. And yet, the European Commission’s European Code of Best Practices Facilitating Access by SMEs to Public Procurement Contracts56 explicitly states the need for procuring bodies within EU Member States to provide SMEs with intelligence as to the strengths and weakness of their bids. It also mentions that in doing so, public bodies demonstrate compliance with the Principle of Transparency,57 as enshrined within the Public Procurement Directives (see infra Part IV).

At a global level, there is also tentative evidence of unsatisfactory practice. For example, in Canada, a 2012 study of participation of SMEs in federal procurement noted that forty-three percent of the three hundred and fifty SME respondents reported that they were unsure as to why they were unsuccessful with their bids.58 However, this report did not provide any additional detail as to their exact concerns with the feedback system.59

Clearly, while the above studies offer some insight into feedback provision, they also present significant limitations preventing us from drawing robust conclusions. All the studies are generic (focusing on the theme of SME barriers to engaging in public sector procurement) rather than specifically focusing on feedback quality; hence, they are of limited use because they only provide “high-level” information. Many of the studies are also now somewhat out- dated and report on practice through SME views and perceptions rather than providing certifiable and tangible evidence of actual practice. This article, how- ever, will move the debate significantly forward by discussing the outcome of a dedicated feedback study which empirically examines actual feedback practice and thus provides scope to change practice.

On this note, analysis of studies relating to public sector practices refer to public sector organisations’ claims their feedback practices constitute a key support mechanism for SMEs, which clearly does not marry well with the findings from surveys undertaken with SMEs (mentioned above). Inevitably, because these studies rely on public procurer claims (without verifiable evidence), reporting bias is likely to be present. Perhaps an even more important distinction to make here is the fact that while procurers are stating they are providing feedback, this does not necessarily mean that this feedback is actually of use to help advance disappointed SMEs’ understanding of why they did not win a bid or proceed to the tender stage. This is the crux of the issue. These studies are therefore extremely limited from the perspective of whether SMEs are genuinely benefitting from their feedback.

Indeed, there also appears to be a major dearth in the academic literature from this perspective. For example, authors such as Flynn and Davis have highlighted in their research the high levels of compliance in Ireland by public procurers in providing feedback (with eighty-eight percent of four hundred and thirty-six public buyer respondents claiming they provide feedback).60 Similarly, Loader reports in an article on local authority support to SMEs in public procurement in England that a high proportion of local government organisations state they are providing feedback to SMEs.61 Again, this research has its limitations from an SME perspective because it relies on self-reported behaviors, and more significantly does not reveal the extent to which this feedback was or was not meaningful to SMEs in terms of enabling them to understand why they have been unsuccessful and hence enable them to reflect upon how they can improve subsequent tenders. Ultimately, the latter is what the various definitions of feedback and references to feedback all have in common from the perspective of what feedback should achieve and arguably represents the “holy grail” of feedback provision. A key line of enquiry that this article will examine is therefore whether feedback in practice is consistent with this aim, or whether this aim is more akin to rhetoric.

IV.  Legal Basis, Explanation, and Analysis

A. Legal Entitlement to Feedback for Above EU Threshold Contracts

Article 55 of the 2014 Public Contracts Directive 2014/24/EU62 provides that when an EU-level contract (i.e., a contract that exceeds the Directive’s financial thresholds) is awarded,63 then the public procurer must issue an Alcatel standstill letter to all losing bidder(s) containing the following information:

•   Name of the winning bidder;

•   The award criteria by which the bids were evaluated; and

•   The score (or more usually scores) of the successful bidder and losing bidder.64

During the standstill period, which is provided for by the Remedies Directive 2007/66/EC,65 the contract cannot be signed with the successful bidder. The standstill period can vary between ten to fifteen days, depending on the circumstances of the means by which the award decision is conveyed.66 During this period, the losing bidder(s) may attempt to seek the pre-contractual remedy of suspension of the award decision in order to prevent the contract being signed.67

However, for present purposes, this article focuses on another course of action the losing bidder(s) may seek to take during the standstill period: the right to request feedback. In this respect, Article 55 of Directive 2014/24/EU consolidates a number of rights for the losing bidder(s), which were set out in previous European Court of Justice/ General Court judgments in recent years.68 Article 55(2) inter alia grants the losing bidder(s) the right to be given the “characteristics and relative advantages” of the winning bid: what this means is that at a very minimum, in addition to the score of the winner (which will have already been disclosed in the award decision notification),69 the losing bidder(s) must be able to understand from the award decision why the winning bid was stronger than theirs. In other words, the award decision must contain at a minimum intelligible information that will enable them to understand why they were not selected as the winning bid. In some cases, the mere disclosing of the final scores will satisfy this requirement of intelligibility. However, there will be occasions where the mere disclosure of the winning and losing bidder’s scores, under the relevant award criteria, will not enable the required level of understanding as to why the winning bid won and the losing bid did not. A typical situation is where the lowest price bidder did not win the contract and instead a higher priced bidder won.70 In such a situation, more narrative is required to help the lowest price bidder understand why they were not awarded the contract. The essence of recent case law is that feedback given must be of such a quality that the losing bidder(s) can understand why their scores were lower than the winner’s scores; so the feedback explanation must be sufficiently specific to achieve this outcome, rather than be generic in nature.71 Therefore, in effect, the losing bidder(s) seeking feedback must be put in a situation where they can compare their performance with that of the winning bidder under each main award criterion.

As alluded to above, the giving of scores without additional narrative is in some cases sufficient to enable the required level of understanding, such as where the losing bidder was awarded lower marks than the winner under the lowest price criterion. That is a net point decision, i.e., one can readily understand that the lowest price bid will get the maximum marks under the “price” criterion and that higher priced bids will merit lower marks, etc. However, when it comes to putting the losing bidder(s) in a position whereby they can understand why their awarded score relative to the winner’s was less generous on a qualitative criterion, then mere revelation of winner and losing bidder scores on their own will not be sufficient, and an additional explanatory narrative must be provided in order to allow the losing bidder to understand why their submission on the point in question was weaker relative to the winner. This approach is a reflection of the EU General Principle of Transparency72 in action and has been stated on multiple occasions in the case law (as set out below).

The foregoing has been confirmed not only by the European Union courts, but also by the national courts in recent times. For example, in the Northern Ireland courts, in McConnell Archive Storage Ltd. v Belfast City Council,73 the Northern Ireland courts made it clear that while the transparency obligation does not require the public procurer to reveal everything about how the bids were conducted and evaluated, it does require the procurer to disclose information sufficient to allow a losing bidder to understand how the winner and the losing bidder’s scores were arrived at. In the Irish Republic, the High Court (a superior court of unlimited jurisdiction) held in RPS Consulting Engineers v. Kildare County Council 74 that the purpose of the standstill period is to allow the losing bidder(s) to request feedback to help them understand how the procurement was conducted; this includes whether they have grounds to challenge the contract being awarded. Hence, the Court held that the procurer had failed to place the losing bidder in this position because it had inadequately responded to a reasonable request for information which would have allowed the bidder to assess whether it had grounds to challenge the contract.75 The context in this case is particularly illuminating: the losing bidder tendered at a lower price, yet did not win the bid; the feedback provided was inadequate, held the Court, because it did not help the losing bidder understand why the procurer chose to select as the winner a higher priced bidder.76 Hence, the procurer had failed in its duty to provide an adequate explanation, which would allow the losing bidder understand both in fact and in law why it did not win the contract.77

Sometimes disclosing scores alone is acceptable, as in Case T-437/05 Brink’s Security Luxembourg SA v. Commission of the European Communities,78 where the General Court’s judgement implies that if the reasons for scoring are not clear, and a comparison is involved with the winner’s scores (in order to aid understanding of the award decision), then the procurer is required to provide a narrative to explain the relative scores. In some cases, it might be possible that disclosing (just) scores can be sufficient where a detailed methodology for reaching scores is also disclosed in such a way that it enables the losing bidder to understand how the scores were reached.79

Finally, sometimes a bidder may wish to know why they were deemed not to have satisfied a particular technical requirement specified in the tender or where they ranked relative to others. In Case T-183/00 Strabag Benelux NV v. Council of the EU,80 the Court held that it is not sufficient to say that a bidder did not meet a specific technical requirement; it must be additionally explained why they failed to meet it so that they understand why they were found not to satisfy it. Similarly, on the question of communicating ranking (i.e., positioning relative to other bidders), this is not an automatic entitlement unless it is necessary for the losing bidder to understand why they lost.81 Sometimes, positioning of the bid in ranking will require explanation in order to enable the losing bidder seeking feedback to understand why they were so ranked.

B.  Legal Entitlement to Feedback for Below EU Threshold Contracts

An interesting question that arises is: what is the entitlement to feedback at the sub-threshold level? The EU Procurement Directives impose a duty to give feedback to disappointed bidders.82 But what of the situation with sub-threshold contracts? While the EU Directives do not impose a statutory duty in the case of sub-threshold level contracts,83 nevertheless it may be said that the duty to give reasons to a non-successful bidder could apply in the case of non-threshold contracts, based on actionable domestic grounds for judicial review within the UK. Although traditionally the view has been that there is no general right in administrative law to be given reasons for a decision,84 judicial review cases have indicated that a public body might have a duty in such circumstances where not doing so would lead to a procedural impropriety. Such a principle has been interpreted by the courts based on the public law proposition that the decision-makers – in this context, the contracting authorities – must act fairly and rationally.85

In R v. Secretary of State, ex parte Doody,86 Lord Mustill set out six principles of fairness in public law decisions. The fifth and sixth are germane to the present discussion:

5.  Fairness will very often require that a person who may be adversely affected by the decision will have an opportunity to make representations on his own behalf either before the decision is taken with a view to producing a favourable result; or after it is taken, with a view to procuring its modification; or both.

6.  Since the person affected usually cannot make worthwhile representations without knowing what factors may weigh against his interests, fairness will very often require that he is informed of the gist of the case which he has to answer.

Applying these principles to the question of whether a disappointed bidder is entitled to feedback of the same quality as in the case of an above threshold level bidder, it is clear that the disappointed bidder must be given enough information to enable them to understand why their bid did not win and to enable them assess if they have been the subject of procedural unfairness (e.g., whether scoring criteria changed during the evaluation process to their detriment or why their bid was not deemed the winner, notwithstanding the fact that they bid the lowest price). Also in line with the EU-level duty, at the sub-threshold level, the level of feedback required will vary from case to case depending on particular circumstances. So, the extent of the obligation to give reasons is one that must be complied with if it is a “common standard” (per Sedley, J. in R (on the application of the Asha Foundation) v. Millennium Commission).87 Thus, a refusal to give reasons for a contract award decision that are not sufficiently detailed so as to place the bidder in a position to assess whether there has been procedural unfairness will incur the wrath of a judge.

In circumstances which lead to procedural unfairness, the courts could find that the failure to give reasons could lead to a finding that the decision was unlawful, rendering it vulnerable to a quashing order (nullifying the original decision) or alternatively, a mandatory order (compelling the public body to justify its decision).88

In conclusion, while there is no statutory right to feedback for sub-threshold contracts (unlike above threshold level contracts), the procurer of such a contract is nevertheless obliged to communicate sufficient feedback to the disappointed bidder to such an extent to assure the bidder there has been no procedural unfairness. The extent of this obligation is illustrated by a recent decision in the Irish High Court, RPS Consulting Engineers v. Kildare County Council,89 where the High Court held that it was unreasonable for the contracting authority to refuse to give reasons (beyond communication of scores) for its award decision. It ordered the contracting authority to supply reasoning within fifteen days to allow the losing bidder to understand the basis for the award decision and to help it decide whether any procedural impropriety had occurred.90

V.   Method

This article draws from evidence collected from delivering the £3.7 million, forty-one month, EU-funded “Winning in Tendering” project’s Tender Review Service (TRS), offered to SMEs over a four year period. Bangor University’s Institute for Competition & Procurement Studies (ICPS) was the lead partner in this strategic project, and the project was partly funded by the European Union’s Ireland-Wales INTERREG Programme.91

The TRS was established following Welsh SME claims, observed by the Bangor University research team during its work on the Barriers to Procurement Opportunity Report,92 that they were not receiving sufficiently meaningful feedback from public procurers on their unsuccessful bid submissions. In order to help SMEs learn how to improve future submissions, the TRS provided a free, impartial, and confidential review and forensic analysis of an unsuccessful tender or pre-qualification questionnaire (PQQ) submitted by Welsh SMEs to UK public sector organisations.93 The service was available to any Welsh SME who had previously submitted a tender or PQQ to the public sector.

SMEs who wished to take advantage of the service were strongly advised to send Bangor University’s ICPS their procurer feedback letters alongside their failed tenders or PQQs, as well as the corresponding invitation to tender documentation. It is these feedback letters, comprising of twenty-eight separate tender feedback letters and eleven PQQ feedback letters, that form the basis of this analysis.94 They account for all the feedback letters received in delivering the TRS in Wales (i.e., a one hundred percent response rate).95

The public procurer feedback letters researched cover eighteen separate public sector organisations from England and Wales and thirty-nine separate procurement exercises and cover a mixture of above and below threshold level contracts.96 They include a diverse mixture of organisations (see Chart 1), including local government (accounting for fifty-nine percent of all feed- back letters examined), central government organisations (accounting for thirty-three percent), the police force (five percent), and universities (three percent).97 They also relate to a number of different industry sectors including construction, business services, environmental, training, manufacturing, printing, and publishing.98

Chart 1 - Proportion of Feedback Letters Examined by Public Sector Organisation Type

This study has adopted content analysis to analyze the information contained within the feedback letters. Content analysis is a suitable approach in this context given that both quantitative and qualitative methods can be used. A quantitative approach is utilized here to determine the frequency of the different feedback components (identified iteratively as each feedback letter was systematically examined). These feedback components are set out in the next section. In contrast, a qualitative approach is utilized to undertake a detailed analysis of evidence relating to actual feedback practice and to assess whether this feedback is truly meaningful to SMEs.

VI.  Findings and Discussion

The following section presents a content-based examination of the thirty-nine feedback letters and any accompanying appendices. The analysis (1) summarizes the frequency of occurrence of the different components of feedback identified within the feedback letters analyzed and; (2) provides illustrations as to the usefulness, or otherwise, of the feedback provided.

In this section, the article divides feedback into four key categories, based on the different components of feedback identified in this article’s study.99 This is the first-ever attempt at categorization of the different components of feedback. These will now be discussed in turn. In Part VII, the article proposes a unique Universal Feedback Methodology that satisfies both SME-friendly procurement (policy objectives) and legal compliance (E.C.J. case law and legislation), regardless of contract size, complexity, or jurisdiction. Public pro- curers who adopt this proposed methodology can be confident that they are operating in an SME-friendly fashion, satisfying transparency and integrity requirements while also being legally compliant and enabling SMEs to defend their position if procedural unfairness has occurred.

A.  Feedback Disclosure Category 1: “Precursory” Feedback Information

i.    Category Description

This feedback disclosure category refers to material given to SMEs within invitation to tender or PQQ documentation – i.e., prior to receiving feedback –  in order to prescriptively furnish them with sufficient information such that they understand at the outset what feedback information they will receive should they be unsuccessful (in terms of general content and extent, as well as delivery format). This category also includes evaluation criteria and scoring methodology information that subsequently provides the context for any feedback provided.

Table 1 provides a summary of the various feedback components within the “precursory” feedback information disclosure category and their frequency of occurrence within the study.

Table 1: Feedback Component Frequency - "Precursory" Feedback Information Summary Statistics100

ii.  Summary Finding

As reflected in Table 1, very rarely are SMEs furnished with information up-front in invitation to tender documentation as to what they should expect to receive in feedback and how it will be delivered. It also shows that there was universal compliance in relation to publishing evaluation criteria at the out- set within a tendering process,101 but that this information – key to enabling SMEs to understand the basis for their scoring – was only sometimes repeated within feedback letters.102 This clearly breaches SME-friendly procurement principles. The final headline finding for this category was that while a scoring methodology (defining, for example, what an “excellent” response is, as opposed to a “good” response) was provided in approximately two-thirds of cases at the outset, this scoring methodology was only very rarely repeated within the feedback letters themselves.103 This suggests poor practice in this regard is the norm. The narrative that follows provides an in-depth discussion of the findings within this category.

iii.  Detailed Analysis

The results in this disclosure category were mixed. Surprisingly, only one contract opportunity (three percent of all those examined in the study) provided an explanation at the outset to give the suppliers an appreciation as to what feedback information they would receive should they be unsuccessful.104 From both a transparency point of view and a best practice perspective, suppliers should be made aware from the very beginning what feedback they can expect. Indeed, it is not an unrealistic proposition to suggest that this information should form part of the wider intelligence SMEs should have access to when deciding whether to bid for a public sector contract or conversely decide to opt out. Inevitably, SMEs, particularly those new to tendering (who are most likely benefit from feedback provided by procurers),105 will be particularly interested in establishing the facts from this perspective before making a decision on whether to participate in the bid.

Another precursor to providing feedback is to state the evaluation/ award criteria at the outset in invitation to tender documentation or PQQ documentation. Fortunately, on this point there was universal compliance (one hundred percent) within the study.106 However, best practice would naturally also extend to repeating this information within feedback letters in order to remind suppliers how their tender or PQQ was evaluated. From this perspective, a decidedly mixed picture emerges, with only forty-nine percent of the feedback letters supplying this important information to SMEs.107

A scoring methodology was provided within procurer’s invitation to tender or PQQ documentation (detailing, for instance, what would constitute an “excellent” response as opposed to a “good”, “average” or “poor” response), in sixty-four percent of cases.108 However, closer examination of these “scoring methodologies” reveals that their value to SMEs was questionable in many cases in terms of the extent to which they might be informative, given that they often contained “high-level” information only. Table 2 below provides two separate examples, with one being a good illustration (i.e., providing insightful information to help guide SME responses) and the other being a weaker (and typical) example that provided much more generic information. With regards to the question of whether scoring methodology information was repeated within feedback letters, this was only evident in one feedback letter, or three percent of cases.109 Again, this is reflective of poor practice.

Table 2: Good & Weak Examples of Scoring Methodologies110

B.   Feedback Disclosure Category 2: “Elementary” Feedback Information

i.    Category Description

The “elementary” feedback disclosure category comprises those components of feedback that arguably are relatively easy for procurers to provide to SMEs because they are typically quantitative in nature (i.e., scoring information) and require a minimal time commitment from the procurer to cite. This disclosure category can be distinguished from “precursory” feedback information on the basis that this category contains information that enables SMEs to first, quickly gain a basic understanding of how they performed (including areas they performed well and less well in); second, gain a general appreciation of how they fared in comparison to others (including a basic understanding of the volume of competition they faced); and third, understand who the winner was.

Table 3 provides a summary of the various feedback components relevant to the study within the “elementary” feedback information disclosure category and their frequency of occurrence.

Table 3: Feedback Component Frequency - "Elementary" Feedback Information Summary Statistics111

ii.  Summary Finding

This information shows that a lack of a standardized approach and a lack of transparency was commonplace within this category. A large proportion of feedback letters did not provide a breakdown of scores per evaluation criteria item.112 A scoring comparison with the winning supplier was provided in some cases and not others, and only relatively rarely in the context of this comparison was it provided for every evaluation criteria item.113 Naming the winning supplier was also by no means a given. The narrative that follows provides a detailed discussion of the findings within this category.

iii.  Detailed Analysis

What the above data reveals makes for unpleasant reading, with a lack of a standardized approach and a lack of transparency being the most immediately noticeable aspects. While provision of an SME’s overall tender or PQQ score was the most commonly cited information, surprisingly, even this was not a given, with eight cases (or twenty one percent of letters) identified where this information was not provided.114 Failing to specify this information puts an SME at a clear disadvantage in terms of gaining a basic understanding of how well or poorly they fared. This failure clearly contravenes EU procurement law. In addition, it could naturally lead to the SME posing questions as to the legitimacy and transparency of the whole evaluation process. This finding is particularly interesting given that the European Commission has recently stated that building trust among market players, including SMEs, to participate in procurement procedures is a key means of improving SME access to public procurement markets.115

For feedback relating to the tender stage, in only seventy-one percent of cases were separate overall scores provided for both the quality element and the price element.116 This meant that some SMEs would have been left “second-guessing” how well they fared for these very distinct elements, both of which are key to competitive positioning in a tender competition. Without this information, SMEs will be left in the doldrums to a considerable degree and unsure of how to progress. A deeper analysis reveals that a detailed breakdown of every individual score for each evaluation criteria item was specified in only forty-three percent of cases.117 This meant that in the remaining fifty-seven percent of feedback letters, there would have been no intelligence provided to enable SMEs to understand which areas they scored strongly or less strongly in, which would have permitted them to concentrate their improvement efforts on the basis of their scoring.

In relation to enabling comparisons on the basis of scoring with the winning supplier/ all suppliers, sixty-four percent of tender feedback letters provided some element of a score comparison with the winning supplier/ all suppliers.118 This meant that thirty-six percent of feedback letters provided no scoring comparison whatsoever to losing bidders to enable them to gauge their performance and understand to what extent they need to improve in the hopes of having a realistic chance of success in future bids.119

Roughly fifty-three percent of those providing a comparison of scoring with the winning supplier/ all suppliers (accounting for thirty-four percent of all tender feedback letters) provided solely the score of the winning supplier/ all suppliers without providing a breakdown of those scores.120 The other forty-seven percent provided both the winning tenderer/ all suppliers overall score and a breakdown of their scores; these accounted for thirty-three percent of all tender feedback letters.121 The latter statistic is deeply concerning because it means the modus operandi is not to provide this information. This is a clear breach of the principles of SME-friendly procurement. It is also worth noting that only thirty-three percent of all tender and PQQ feedback letters indicated the overall position of the SME in comparison to their competitors122  – information that arguably could have been provided with ease.

Given that the EU Procurement Directives state that the successful tenderer must be named, it was most surprising and somewhat unnerving that in only fifty-nine percent of cases was the name of the winning supplier mentioned in the tender feedback letters.

With regard to the level of competition faced, the number of tender or PQQ submissions received by the public sector organisations was stated within only thirty-one percent of the feedback letters examined.123 Again, this is information that could have been provided with minimal difficulty.

C.  Feedback Disclosure Category 3: “Core” Feedback Information

i.    Category Description

The “core” feedback information disclosure category includes those aspects of feedback that potentially present an SME with clear reasons as to why they were not successful and thereby helps them understand how to improve things for next time. This disclosure category is usually provided in narrative (qualitative) format rather than in quantitative fashion (as in the case of “elementary” feedback information). From a practical point of view, procurers might spend more time preparing this information than Disclosure Category 2; however, from an SME perspective, it is this insight that has the potential to benefit and enlighten them most.

Table 4 provides a summary of the various feedback components relevant to the study within the “core” feedback information disclosure category as well as their frequency of occurrence.

Table 4: Feedback Component Frequency - "Core" Feedback Information Summary Statistics124

ii.  Summary Finding

The information shows that providing narrative commentary as to why a supplier was unsuccessful is by no means a given (it occurred in just under half of cases) and where it is provided, it is delivered in differing ways and can still be non-transparent where the explanations given are too “high-level.” Only one in ten feedback letters gave an explanation of the score for each evaluation criteria item and positioned these comments in relation to the winner in each case.125 The narrative that follows provides a detailed discussion of the findings within this category.

iii.   Detailed Analysis

The findings in feedback disclosure Category 3 (“core” feedback information) were eye-opening from a number of different perspectives. Incredibly, only forty-nine percent of the feedback letters detailed any narrative at all in relation to why an SME was unsuccessful.126 In the remaining fifty-one percent of cases, typically all that was provided was their final score in comparison to the winner’s score or the PQQ threshold score or a breakdown of these scores per evaluation criteria item.127 For example, one “feedback” letter, simply stated: “[y]our final score is 54.20% and the pass mark is [sixty], therefore you have not been successful on this occasion.”128

On the basis of the definitions and references to feedback set out in the introduction to this article, this would not constitute feedback because it does not enlighten the SME as to where they went wrong and hence how they can improve. This is another vivid example of a clear breach of the principles of SME-friendly procurement. In those cases where narrative was provided, the research demonstrated qualitatively that feedback was delivered in one of three main ways.

a. Method 1

Method 1 is: providing general commentary as to why the SME was unsuccessful (with potentially some positioning in the narrative in relation to the winning supplier) but failing to provide specific reference to each evaluation criteria item. The problem with this approach is that there is a risk that the feedback may only pinpoint one or two weaknesses without revealing other deficiencies related to separate evaluation criteria items that in theory could prevent SMEs from winning future contracts. Additionally, from a legal perspective, it places SMEs in a position whereby they may not be able to ascertain if a procedural unfairness has occurred in the evaluation process. This is because without being provided with narrative commentary as to the “characteristics and relative advantages” of the winning bidder’s submission (and simultaneously being furnished with both the SME’s scores and the winner’s scores for each evaluation criterion item), the procurer may have failed to place the SME in a position that they be enabled to understand the “characteristics and relative advantages” of the winning bid.129

This approach accounted for forty-three percent of all narratives provided and twenty-one percent of all the feedback letters examined.130 In practice, this was administered in different ways. For instance, one feedback letter contained three “highlights” and three “lowlights” of the submission, which provided a useful meaningful steer as to how that SME could improve.131 In other cases, a simple one-line explanation was given. For example, one stated: “[t]he company who was awarded the contract scored higher in the methodology/  proposal section.”132

While this provides the SME with at least some indication as to where they were less strong than the ultimate winner, in reality this narrative provides no additional benefit beyond specifying the suppliers range of scores for different evaluation criteria items vis a vis the winning supplier’s scores (i.e., Disclosure Category 2: “elementary” feedback information).

Had this feedback been in alignment with the feedback definition/ references provided in the introduction to this article, then a much more prescriptive comment would have been made, stating perhaps that the tender was missing core methodology statement information and providing some examples of the same. Arguably, the value of this feedback would then have become exponentially more useful to the supplier, elevating it to the status of being consistent with the principles of SME-friendly procurement.

A different example from a separate feedback letter is similarly unhelpful to SMEs. It stated that the reasons for the decision, including the characteristics and relative advantages of the winning tender, were: “[t]he quality element of the successful tender paid full attention to all aspects of the technical questions asked and showed evidence of detailed attention to the requirements of this specific project in the answers.”133

In contrast, other feedback letters gave detailed reasoning as to how the supplier fell short, providing them with a definitive compass as to how they could improve future submissions.134

b. Method 2

Method 2 is: providing an explanation of the score given for each evaluation criteria item. This is better than Method 1 in theory because it is more likely to provide holistic feedback because all evaluation criteria items will be addressed.

This delivery mechanism accounted for thirty-seven percent of all narratives provided and eighteen percent of all feedback letters examined.135 Once again, the research demonstrated that the actual value of this feedback would differ markedly. For instance, in some cases,136 the narrative comment for each evaluation criteria item was so high-level in nature (e.g., “meets some of the requirement”) that it could arguably be deemed meaningless. Notwithstanding this, there were examples in this study where very simplistic commentary of this manner was combined with very detailed and prescriptive indicative content for each evaluation criteria item (i.e., common content included within feedback letters to all suppliers, providing an indication of what the evaluators were looking for).137 Where this approach was taken, it was evident that this could enlighten suppliers as to how to improve things for next time.138

c.  Method 3

Method 3 is: providing commentary explaining the score for each evaluation criteria item and positioning these comments in relation to the winner in each case to provide the necessary context. This method is best in theory because explanations will be provided per evaluation criteria item and the SME will have contextual understanding as to the reasons for their score relative to the winner. The latter aspect is also key in reassuring SMEs that the procurement process was conducted with integrity and hence builds trust between procurers and SMEs.

This method accounted for twenty percent of all narratives provided, but incredibly, only ten percent of all feedback letters examined took this approach.139 Again, the most striking aspect in evaluating these feedback letters was the fact that the narratives varied significantly in terms of whether it was useful or not to an SME. In some cases, there was no additional benefit for SMEs in communicating this information in such a way. For example, one feedback letter’s explanation for an SME’s score for one particular evaluation criteria item was: “[s]atisfactory response, but doubts on several aspects.”140 In terms of reasoning for the award of score to the winning supplier for the same evaluation criteria item, the following explanation was given: “[m]eets all requirements in a satisfactory manner.”141 This example thus illustrates that even when Method 3 (arguably the method with most potential to benefit suppliers) is used to provide feedback, it is still entirely possible for the principles of transparency and SME-friendly procurement to be blatantly breached.

In contrast, another feedback letter provided extremely useful information to a supplier and is an example of good practice.142 This information included:

•    The evaluation criteria and weightings;

•    Scores for each evaluation criteria item (including price), alongside the winning supplier’s score in each case;

•    Overall score, alongside the winning supplier's overall score;

•    Detailed reasons for the decision in relation to each evaluation criteria item (totalling half a page of content);

•   A clear description of the characteristics of the successful tender (total- ling a third of a page of content); and

•   The relative advantages of the successful bidder (totalling a quarter of a page of content).

D.  Feedback Disclosure Category 4: “Follow-up” Feedback Information

i. Category Description

This final feedback disclosure category simply relates to procurers offering in their feedback letters the opportunity for additional clarification or more detailed feedback, either through written feedback or subsequent telephone or face-to-face feedback or a combination of these methods.

Table 5 provides details of the frequency of occurrence within the study of the solitary feedback component within the “follow-up” feedback information disclosure category. The narrative that follows discusses the findings within this category.

Table 5: Feedback Component Frequency - "Follow-up" Feedback Information Summary Statisticss143

Only just over half (fifty-one percent) of the feedback letters made reference to the opportunity for additional clarification or more detailed feedback.144 This is most surprising, especially given that the feedback in some cases within the letters contained little to no useful information to reveal to disappointed SMEs where they went wrong. This is particularly concerning from the perspective of complying with the principles of SME-friendly procurement because it suggests that not only are procurers in some cases failing to enlighten SMEs as to the strengths and weaknesses of their submissions, but that they are also reticent to engage in additional dialogue with SMEs in order to provide this opportunity for enlightenment.

VII.  Implicatins for Policy and Practice

By drawing on empirical evidence, this article has examined actual public sector procurement feedback practice via the lens of whether current feedback practices are meeting the broader needs of SMEs.145 This study has taken place within the wider context of numerous jurisdictions seemingly taking a proactive stance towards embracing SME-friendly procurement and being vocal in adopting policy in this area.

Given that both quantitative and qualitative methods have been used to analyze the feedback letters, this exercise has generated a rich variety of data against which to assess implications for policy and practice and provides clear evidence to suggest that feedback is “unloved” in the context of SME-friendly procurement while simultaneously raising serious questions as to why this might be the case. The key inferences for policy and practice to be drawn from the study are discussed below.

A.  Consistency and Oversight of Feedback Provision

The most striking aspect of the feedback letters was the distinct lack of consistency in terms of the information provided within each letter, with the breadth, depth, and meaningfulness of the information provided varying significantly from one feedback letter to another and often proving non-insightful from the SME’s perspective.146

This “free-for-all” suggests there is currently insufficient oversight of feedback provision by the governmental bodies responsible for best practice procurement provisions and for setting procurement policy. Indeed, an initial analysis of up-to-date best practice feedback guidance within the UK147 suggests that it is anaemic, focusing mainly on legal imperatives and Alcatel letter templates and thus fails to provide sufficient depth to inform and educate procurers as to what they should be doing from an SME perspective. This void is not unique to the UK: on the international stage, only one document stands out as being especially useful, and this is an Office of the Procurement Ombudsman review of supplier debriefings in Canada148 - and even this document is now ten years old and provides little context with regard to SMEs.

It is important to recognize that as a result of the vulnerabilities they face, SMEs can be disproportionately impacted by the poor feedback practices that have become evident in this study when compared to large businesses. These vulnerabilities include the fact that they often suffer from having a limited resource base149 and typically do not have access to tendering specialists,150 the result being that they are less likely to be aware of their deficiencies and how to “put them right.” Therefore, SMEs potentially have more to gain when feedback is provided in a meaningful way because feedback can provide them with a definitive compass as to how they can improve.

B.  Legal Compliance Versus SME Needs

Earlier this article examined the non-imperative approach that SME-friendly procurement policies take when “encouraging” public bodies to provide meaningful feedback.151 This article’s findings on this aspect are that the evidence suggests that legal compliance152 and the risk of legal challenge is proving the primary driver in feedback practice rather than feedback being an opportunity for suppliers to learn and subsequently improve future bids. This suggests the needs of SMEs are being put firmly on the back burner. On this note, interestingly, of the sixteen recommendations made in the Barriers to Procurement Opportunity Report153 to remove barriers to Welsh SMEs engaging in public sector procurement, the provision of meaningful high-quality feedback to all suppliers was the only key recommendation that has not been implemented by the Welsh government.154 The legitimacy of the claim that legal compliance “culture” and the risk of legal challenge are the key reasons why SME needs have been traditionally de-prioritized in the context of feedback should be the subject of additional future research.

C.  A Universal Feedback Methodology

This study demonstrates that recent calls by the UK Federation of Small Businesses155 to publish an action plan showing how they will enforce the provision of more detailed feedback are not without merit. The research has shown that the UK government could do worse than give these considerations serious thought, particularly if the government intends to achieve its goal of thirty-three percent of its procured spend to be with SMEs by 2022.156

The empirical evidence emanating from this research has led to the development of the first-ever categorization of different feedback components (see Figure 1 below) and the creation of a unique Universal Feedback Methodology (see Table 6 below) that could facilitate a transformation in approach to feedback not just in the UK, but elsewhere as well.

In this section, the article sets out four essential elements of feedback (see Figure 1 below). It then describes (see Table 6 below) the proposed Universal Feedback Methodology, which provides a standardized157 approach to feedback provision regardless of contract size, complexity, and jurisdiction. The methodology will simultaneously ensure that public procurers comply with the EU policy of SME-friendly procurement and provide legally sound feedback, thus satisfying transparency principles and the sensitivities of legal compliance “culture.” This methodology, therefore, presents an opportunity to realign feedback practice – which this article’s study has shown does not meet the needs of SMEs –  with SME-friendly procurement principles that do. The Universal Feedback Methodology also presents an opportunity to ensure that each time a tenderer receives feedback, they receive it in a broadly similar format in terms of feedback content, feedback intent, and feedback extent regardless of jurisdiction (see Table 6), something that this article’s research has shown has been severely lacking to date. It is important to make the point that while this feedback methodology has been designed to minimize the risk of SMEs being disproportionately impacted by poor feedback practice (because of their limited resource base),158 this article proposes that the Universal Feedback Methodology is to be used for feedback provision to all businesses, regardless of size.

This article asserts that this written feedback, detailed in Table 6, should be provided routinely in a contract award letter and not only on request from the disappointed bidder following a contract award notification (which is what existing EU procurement law stipulates).

Figure 1: The 4 Essential Elements of Feedback by Feedback Disclosure Category

Feedback Disclosure Category 1 (“precursory” feedback information) is warranted because the research159 has shown that SMEs are rarely informed within invitation to tender documentation what feedback information they will receive should they be unsuccessful (in terms of general content, delivery format, and extent). For SMEs, this information might be pivotal for them in deciding whether to opt in or opt out of a procurement opportunity in the first place. Moreover, to provide this information is arguably merely to comply with the EU principles of transparency. Detailing the evaluation criteria is also a legal requirement,160 and this information (alongside scoring methodology information) provides the necessary context to SMEs for any feed- back provided. It also gives them confidence that a fair process was followed, thereby encouraging them to bid in the future.161

Feedback Disclosure Category 2 (“elementary” feedback information) involves the provision of mainly quantitative “scoring” information to enable SMEs to gain a basic feel for how they performed in different areas and how they fared in comparison to others for each evaluation criteria item; it also includes naming the winning supplier and detailing the number of submissions received. This article’s research showed that there were a number of examples where even basic information, such as an SMEs overall score or the name of the winning bidder, was not provided.162 This is in breach of transparency principles and obligations, not to mention being a clear breach of the disclosure requirements for contract award letters. Under half of feedback letters examined provided a breakdown of scores for each individual evaluation criteria item, with less than a third providing both the SMEs’ score for each evaluation criteria item and the winning suppliers scores.163 The methodology proposes that these feedback components are essential requirements because first, they satisfy SMEs’ needs in terms of identifying those areas they need to prioritize for improvement; second, they permit them to gauge the extent to which they need to improve in these areas; and third, by being transparent in providing this information, procurers enhance the trust SMEs have that the public procurement process was conducted with integrity.164 Providing this information also satisfies the EU Public Procurement Directive,165 and EU Case law,166 and the principle of transparency.167

Feedback Disclosure Category 3 (“core” feedback information) has the potential to benefit SMEs most because it provides them with clear reasons (qualitative information) as to why they were unsuccessful, thereby complying with the European Commission’s European Code of Best Practices Facilitating SME Access to Public Procurement Contracts168 and helping them understand how they can improve things for next time. This narrative should provide SMEs with an explanation of the score provided for each evaluation criteria item, ideally with this explanation positioned against the winner in each criterion169 (only one in ten feedback letters in this article’s research achieved this). The information in this disclosure category is essential for SMEs because it removes SME barriers to entry to the public sector marketplace and gives them a better understanding of what is required for them to be successful.170 It also satisfies EU171 and UK172 case law given that it will explain the basis of a public body’s decision to award a contract to one bidder over another, and separately, it enables the losing bidder to defend their rights if procedural unfairness has occurred. It simultaneously satisfies the European Commission procurement strategy related to building trust between SMEs and procurers.173 Adopting this standardized approach to provision of meaningful feedback will also encourage SMEs to bid for cross-border contracts, thus satisfying European Commission policy objectives.174

Feedback Disclosure Category 4 (“follow-up” feedback information) provides an opportunity to satisfy SME needs in cases where the explanatory reasons provided in a feedback letter are not considered by an SME to be sufficiently clear. It does so via the offer within a feedback letter to provide additional clarification. With SMEs historically reluctant to question or seek additional clarification from procurers for fear of jeopardizing their chances of gaining future work,175 this article argues that this opportunity should become standard practice, thereby ensuring SMEs are not fearful of requesting it. In this article’s research, only around half of feedback letters offered this provision.176 This disclosure category acknowledges SME input and time devoted to developing their bids and helps reassure SMEs that the correct process was followed and that it was conducted with integrity.177 Providing this opportunity for additional clarification should, however, not be used as an excuse by procurers to provide anaemic feedback, with the offer of additional clarification giving them a “get out of jail” card. This article submits that the Universal Feedback Methodology should be the norm and should be provided routinely when a contract award announcement is made and not only on request following such notification.

The following table sets out the Universal Feedback Methodology that satisfies both SME-friendly procurement policy objectives and legal compliance with both E.C.J. case law and legislation, regardless of contract size, complexity, or jurisdiction. Public procurers who adopt this proposed methodology will be confident that they are operating in an SME-friendly fashion, satisfying transparency and integrity requirements, whilst simultaneously enabling SMEs to defend their position if procedural unfairness has occurred.

Table 6: Universal Feedback Methodology which can be applied to All Complexity, and Jurisdiction

D.  Re-engineering Feedback Provision - Implementation Considerations and Possible Unintended Consequences

On a practical implementation level, the issue of access to resources to facilitate change warrants due consideration and may be a barrier to change. However, this research has demonstrated that there are ways and means to provide feedback in an efficient and “leaner” way and still have the feedback retain its effectiveness in acting as a catalyst in enabling SMEs to improve future submissions. One such example, unearthed in this study (see supra Part VI)190 involves combining simplistic “high-level” commentary on a bespoke level to suppliers such as “meets some of the requirement” with detailed and prescriptive indicative content for each evaluation criteria item. This example is useful because it serves to illustrate that legally sound avenues do exist191 to combine efficiency and effectiveness and serves as a reminder that they are not mutually exclusive.

In addition, there are aspects of feedback that should be quick and easy to provide to SMEs (assuming the evaluation process has been carried out in a proper and methodological way). These aspects relate in particular to Feedback Disclosure Category 2 (“elementary” feedback information). It is also important to make the distinction that, while it is vital for an SME to be given clear reasons as to why they have not been successful in the “core” feed- back information category, it is neither fair, nor realistic, for parties to expect feedback to extend into the domain of supplier development activity.192 This activity is usually undertaken by economic development agencies and is an entirely separate activity and should not form part of public procurers’ already indisputably diverse role.

In countries where devolved procurement is more common than a centralized model, it is also important to recognize that procurers may not be professionally trained and that procurement may be just one of a number of roles that they perform.193 Within such an environment, as concluded by Preuss,194 a decentralized procurement model can prove to be a challenge to the implementation of procurement policy. Hence, the difficulty of achieving wholescale changes and achieving compliance with a feedback methodology such as that set out in Table 6, particularly in an era of austerity, should not be underestimated. Indeed, this research has demonstrated that there were even significant differences in feedback practice within the same public sector organisation on occasion, with the decentralized approach to procurement certainly having a major impact on this.

The aforementioned example highlights that where procurers are not professionally trained, the risks to the public sector from adopting a more transparent approach to feedback provision are more acute. For instance, in such a scenario, there might be a heightened likelihood that a supplier could challenge the feedback provided, perhaps on the basis that some of the narrative could be perceived as being subjective in nature. This illustration thus draws attention to the fact that all those personnel involved in the evaluation process and the giving of feedback should receive formal training in this area. The benefits of such an approach are threefold: first, it will enable them to have a full understanding of how to provide effective feedback; second, it will help them fully comprehend the wider repercussions of providing feedback lacking in clarity; and third, it will give them a strong understanding of what information relating to the winning bidder is allowed to be shared with the unsuccessful bidder, particularly from the perspective of not giving away confidential or commercially sensitive information. In addition to this training, there is also the argument (particularly where those providing feedback do not have professional procurement qualifications), that there should be greater oversight of any feedback provision to tenderers (see Part VII(A)) prior to it being released.

Of course, there are other potential unintended consequences of providing feedback in the manner advocated in this article; these primarily relate to the risks of diverting energies and effort away from other core activities procurers undertake as part of their day-to-day roles. Nonetheless, this article argues that provision of meaningful feedback that helps SMEs and large businesses alike understand how they can improve their future bid submissions and helps build trust between both parties is a key remit of a public procurer.

As a final point, it is important not to underestimate the scale of the challenge in changing public bodies’ procurement culture, i.e., moving from the current, typically conservative feedback culture of focusing on providing the minimal feedback necessary to ensure legal compliance to a more open culture, as proposed in the Universal Feedback Methodology. This challenge is potentially a major barrier to change. However, it is not insurmountable, and the temptation to “do nothing” must be resisted. 195

VIII.  Conclusion

This article’s research has demonstrated that studies that conclude that public bodies are being SME-friendly when it comes to feedback on the basis of quantitative data relating to the proportion of public sector organisations who provide feedback are deeply flawed.196 This article has shown that when it comes to feedback and its usefulness to SMEs, it is very much a case of the old proverb “the proof of the pudding is in the eating.” This finding was acutely evident when analyzing the results from the “core” feedback information dis closure category. Here, there were many examples of procurers in principle providing narrative comments as to the merits of an SME`s submission, but in practice, this information was so “high-level” that ultimately it served very little strategic purpose for them.197

Given that this research has unearthed a plethora of examples of poor feedback,198 but much less in the way of best practice, it has undoubtedly raised awareness of the need for formal feedback audits and for greater scrutiny of feedback provision on a national and international basis, particularly in the context of SME-friendly procurement. While previous academic research has provided evidence to suggest there is a policy-practice divide at a general level with regard to SME-friendly procurement,199 this article has provided substantial weight towards the argument that feedback is very much the neglected child of SME-friendly procurement practice and that SMEs receiving feed- back should be the subject of “tough love,” rather than unhelpful “no love” feedback, with clear, succinct, and informative feedback enlightening them as to how they can improve.

Provision of feedback in this transparent way provides a unique opportunity to enhance dialogue between procurers and SMEs and to build trust between both parties. With the issue of trust recently singled out by the European Commission as being instrumental in improving access to public procurement markets for SMEs,200 this article asserts that it is a major flaw in Commission policy not to explicitly mention the fundamental importance of feedback in enhancing trust. Once again, this raises the ugly question of whether legal compliance (the need for a public body to explain the decision to award a contract to one bidder over another) rather than SME needs (the need to receive high quality feedback on their bid that they expend much time and energy compiling) is taking precedence.

To address this issue, this article proposed a standardized Universal Feedback Methodology that presents an opportunity to realign feedback practice with SME-friendly procurement principles that meet the needs of SMEs. Public procurers who adopt this proposed methodology will be confident that they are operating in an SME-friendly fashion, satisfying transparency and integrity requirements, while remaining legally compliant and enabling SMEs to defend their position if procedural unfairness has occurred.

By aligning feedback quality with both SME-friendly procurement principles and EU and national law compliance requirements in this way, it could transform feedback practices, in particular within those jurisdictions that are adopting a proactive stance in relation to SME-friendly procurement.

Entity:
Topic:
  1. European Commission is abbreviated in the footnote citations of this article as Comm’n.
  2. SMEs are referred to in this article as defined in this Commission Recommendation (defining SMEs as “enterprises which employ fewer than 250 persons and which have an annual turnover not exceeding EUR 50 million, and/or an annual balance sheet total not exceeding EUR 43 million.”). Comm’n Recommendation 2003/361 of 6 May 2003 concerning the definition of micro, small and medium-sized enterprises, 2003 O.J. (L 124) 36, 39.
  3. See infra Part II(A).
  4. This article therefore responds to an acute lack of evidence on this topic and provides a new dataset of information, with a rich variety of detail unavailable elsewhere, in either national or international reports, or in literature on the topic of feedback to SMEs in public procurement.
  5. See infra Part II(B).
  6. These feedback letters were collected as part of the research on the £3.7 million, EU-funded “Winning in Tendering” project’s Tender Review Service, offered solely to SMEs over a 4 year period. Bangor University’s Institute for Competition & Procurement Studies (ICPS) was the lead partner in this strategic project, with the project being part-funded by the European Union’s Ireland-Wales INTERREG Programme. ‘Winning in Tendering’ Public Procurement Research Project, Bangor U., Bangor L. Sch., https://www.bangor.ac.uk/law/winningintendering.php.en [https://perma.cc/2U6B-AHYB] (last visited Apr. 21, 2019) [hereinafter ‘Winning in Tendering’].
  7. Because the analysis discussed in this article involved a confidential review of unsuccessful tenders and PQQs, the study information is not publicly available. Feedback letters and study data are on file with the authors.
  8. Study information on file with authors. Current perspectives into this topic (both from national and international reports and literature) is only “high-level” in nature, and the topic often features as “one of many” components of wider research into barriers to SMEs’ participation in public sector procurement or in research examining methods used to support SMEs in public procurement. These existing studies report on anecdotal practice through two primary mechanisms: firstly, by relying on SME observations and perceptions of the feedback they have received (without empirically assessing the actual feedback content); and secondly, by relying on procurer claims about their feedback practices (again without any empirical examination of these claims). With regard to the latter, some quantitative data exists relating to the proportion of public sector organizations who state they provide feedback to SMEs, but ultimately this data is of minimal benefit because it provides no evidence of whether this feedback is actually useful in giving the disappointed SME an idea of why they lost the bid.
  9. These measures include such things as division of contracts into lots, training, and workshops for SMEs on accessing and winning tender opportunities, and simplified administrative procedures. Org. for Econ. Co-operation & Dev., Government at a Glance 2013, at 136 (2013).
  10. Id. at 42.
  11. The following were adopted as part of the 2014 reforms: Directive 2014/25/EU of the Eur. Parliament and of the Council of 26 Feb. 2014 on procurement by entities operating in the water, energy, transport and postal serv. sectors and repealing Directive 2004/17/EC, 2014 O.J. (L 94) 243; Directive 2014/24/EU of the Eur. Parliament and of the Council of 26 Feb. 2014 on pub. procurement and repealing Directive 2004/18/EC, 2014 O.J. (L 94) 65 [hereinafter Directive 2014/24]; and Directive 2014/23/EU, of the Eur. Parliament and of the Council of 26 Feb. 2014 on the award of concession cont., 2014 O.J. (L 94) 1.
  12. They include limiting turnover requirements to participate in a public procurement procedure, simplifying documentation requirements, and encouraging contracting bodies to divide contracts into lots. E.g., Directive 2014/24, supra note 11, at 65, 79, 81, 129.
  13. Commc’n from the Comm’n to the Eur. Parliament, the Council, the Eur. Econ. and Social Comm. and the Comm. of the Regions: Making Pub. Procurement work in and for Eur. COM (2017) 572 Final 7, 9 (2017) [hereinafter Commc’n from the Comm’n].
  14. Cabinet Off., Cent. Gov’t Direct and Indirect Spend with Small and Medium sized Enter., 2015/16, www.gov.uk/government/uploads/system/uploads/attachment_data/file/650797/Central_Government_Direct_and_Indirect_Spend_with_Small_and_Medium_sized_Enterprises_2015-16_CC.pdf [https://perma.cc/7PV2-UKR6] (last visited Jan. 22, 2018) [hereinafter Cent. Gov’t Direct and Indirect Spend].
  15. These initiatives include the elimination of pre-qualification questionnaires for central government contracts below the EU thresholds and the establishment of the Contracts Finder website to allow suppliers, including SMEs, to search for government opportunities above £10,000. Procurement Policy Note—Update, Cabinet Off. (Apr. 18, 2011), https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/62064/ppn-update-april-2011.pdf [https://perma.cc/65A7-WYFP].
  16. See, e.g., Directive 2014/24, supra note 11, at 65, 99, 214. Above EU threshold contracts refer to services and supplies contracts valued at or above €134,000 for central government contracts and €207,000 for all other contracting authorities and works contracts valued at or above €5,186,000. These contracts are required to be advertised in the Official Journal of the European Union and must be procured according to the EU Public Procurement Directives. Below EU threshold contracts refer to contracts below these values and are not required to be advertised in the Official Journal of the European Union nor procured according to the EU Public Procurement Directives.
  17. PwC, SMEs’ access to public procurement markets and aggregation of demand in the EU 5, 47 (2014) [hereinafter PwC].
  18. Commc’n from the Comm’n, supra note 13, at 5.
  19. PwC, supra note 17, at 5.
  20. Jan de Kok, et al., Do SMEs Create More and Better Jobs? 6 (2011).
  21. Dep’t for Bus., Energy & Indus. Strategy, Longitudinal Small Business Survey Year 2 (2016) SME Employers Cross-Sectional Report 121 (2017).
  22. See generally Jani Saastamoinen, et al., The Role of Training in Dismantling Barriers to SME Participation in Public Procurement, 17 J. Pub. Procurement 1, 3–5 (2017).
  23. E.g., GHK, Evaluation of SMES’ Access to Public Procurement Markets in the EU 57–58 (2010) [hereinafter GHK]; Off. of Gov’t Commerce, Smaller Supplier…Better Value? The Value for Money That Small Firms Can Offer 6 (2005).
  24. Public Procurement Strategy, Eur. Comm’n, http://ec.europa.eu/growth/single-market/public-procurement/strategy_en [https://perma.cc/8B5M-MZVK] (last visited Jan. 22, 2018).
  25. E.g., Off. of Gov’t Com., Supplier Debriefing Guidance 6 (2003), available at https://prp.gov.wales/docs/prp/generalgoodsservices/130617ogcsupplierdebrief.pdf [https://perma.cc /SSJ5-V5UR] [hereinafter Supplier debriefing guidance].
  26. Id.
  27. Id.
  28. See id.
  29. Cf. Sara Carter, et al., Enterprise and Small Business: Principles, Practice and Policy 111–16, 299 (2006) (discussing small business resource constraints).
  30. Supplier Debriefing Guidance, supra note 25.
  31. See id.
  32. Feedback requirements from a legal perspective are discussed in infra Part IV.
  33. Jan Zeber, TaxPayers’ Alliance, Performing Public Procurement 10 (2017) (citing The UK Pub. Sector’s Procurement Process Is the Most Expensive in the Eur. Union, Ctr. for Econ. & Bus. Research (July 11, 2013), https://cebr.com/reports/uk-procurement-most-expensive-in-eu/) [https://perma.cc/4GFH-FJ4W].
  34. See Ruth Fee, et al., SMEs and Government Purchasing in Northern Ireland: Problems and Opportunities, 14 Eur. Bus. Rev. 326, 329 (2002).
  35. E.g., Anthony Flynn et al., Mapping Public Procurement in Ireland, 2 Pub. Procurement L. Rev. 74, 89 (2013) [hereinafter Flynn].
  36. Cf. Supplier Debriefing Guidance, supra note 25, at 7.
  37. E.g., id.
  38. E.g., Lutz Preuss, On the contribution of public procurement to entrepreneurship and small business policy, 23 Entrepreneurship & Reg’l Dev. 787, 797 (2011) [hereinafter Preuss]; Jurong Zheng, et al., The Role of SMEs in public procurement: a Review of Literature and Research Agenda 9–10 (2006) (discussing Northern Irish and South African SMEs and the barriers they face).
  39. Jane Hutt, Welsh Minister for Fin. and Gov’t Bus., Wales Procurement Policy Statement, (June 9, 2015), available at https://gweddill.gov.wales/docs/prp/toolkit/june15walesprocurementpolicystatement2015v1.pdf [https://perma.cc/Y8FQ-MD6U].
  40. Id. at 7–8.
  41. Gov’t of Ir., Dep’t of Pub. Expenditure & Reform, Circular 10/14: Initiatives To Assist SMEs In Public Procurement (2014), available at http://etenders.gov.ie/Media/Default/SiteContent/LegislationGuides/Circular_10_-_14_0.pdf, [https://perma.cc/9QKN-473W].
  42. Commc’n from the Comm’n, supra note 13, at 9–10.
  43. Commc’n from the Comm’n, supra note 13, at 10.
  44. European Code of Best Practices Facilitating Access by SMEs to Public Procurement Contracts, SEC (2008) 12 [hereinafter European Code of Best Practices]. Note, however, that it is important to recognize that the European Code of Best Practices explicitly states the need for procuring bodies within EU Member States to provide SMEs with intelligence as to the strengths and weakness of their bids. Id. at 12; see also infra Part III(B).
  45. Dermot Cahill et al., Barriers to Procurement Opportunity Research (2009), available at http://gov.wales/docs/dpsp/publications/valuewales/barrierstoprocurementopportunity/090520barrierstoprocurement2en.pdf [https://perma.cc/CDD6-26R8] [hereinafter Cahill].
  46. Id. at 9, 38. This research, however, has some limitations from the perspective that it focused on identifying general barriers to procurement for SMEs within the pre-qualification stage only (i.e., it did not specifically address tender feedback) and was also reliant upon SME claims as opposed to actual evidence. Id. at 9.
  47. Anne Glover, HM Treasury, Accelerating the SME Economic Engine: Through Transparent, Simple and Strategic Procurement 56–57 (2008) [hereinafter Glover].
  48. Id. Again, the findings in this report arose from a generic study to understand more about the SME experience of engaging in public sector procurement and failed to provide any additional detail beyond identifying poor feedback as one of their most negative aspects of their engagement with the public sector (seventeen percent of the 873 SME respondents stated this). Id. at 55–57.
  49. Fed’n of Small Bus., Unstacking the Deck: Balancing the Public Procurement Odds 5 (2017) [hereinafter Fed’n of Small Bus.].
  50. Id. at 6.
  51. Id. at 5.
  52. The Crown Commercial Service is the body sitting within the UK Cabinet Office responsible for improving government commercial and procurement activity. Crown Commercial Service, Gov.uk https://www.gov.uk/government/organisations/crown-commercial-service [https://perma.cc/56P9-3MPJ] (last visited May 13, 2019).
  53. Mystery Shopper Publication Table April-June 2017, Crown Com. Serv., https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/634143/Mystery_Shopper_Publication_Table_April_-_June_2017__13_.pdf [https://perma.cc/WUL8-BUKK] (last visited Jan 22, 2019).
  54. One of the recommendations included sharing what their feedback arrangements are within any tender documentation so as to provide suppliers clarity in terms of the feedback they will receive at the end of the tender process. Id.
  55. GHK, supra note 23, at 50–51.
  56. European Code of Best Practices, supra note 44.
  57. Id.
  58. Off. of Small and Medium Enter. and Strategic Engagement Acquisitions, Integral to the Economy, Integral to Procurement, 2012 Study of Participation of Small and Medium Enterprises in Federal Procurement 14 (2013), available at http://publications.gc.ca/collections/collection_2013/tpsgc-pwgsc/P4-55-2013-eng.pdf [https://perma.cc/SHL8-KHGE].
  59. See generally id.
  60. Anthony Flynn & Paul Davis, The Policy-Practice Divide and SME-Friendly Public Procurement, 34 Env’t and Plan. C: Gov’t and Pol’y 559, 565–68 (2016) [hereinafter Policy-Practice Divide].
  61. See Kim Loader, Is Local Authority Procurement Supporting SMEs? An Analysis of Practice in English Local Authorities, 42(3) Loc. Gov’t Stud. 464, 475 (2016).
  62. Directive 2014/24, supra note 11, at 126; implemented domestically via The Pub. Cont. Reg. 2015, SI 2015/102, art. 55 (Eng., Wales, and N. Ir.) and The Pub. Cont. (Scot.) Reg. 2015, SI 2015/446, Explanatory Note 115 (Scot.).
  63. See Directive 2014/24, supra note 11, at 126.
  64. Case C-81/98 Alcatel Austria AG and Others v. Bundesministerium für Wissenscahft und Verkehr, 1999 E.C.R I-7708-09, I-7711-12; see also Birkett Long Solicitors: Procurement Law Jargon Buster – “Standstill Periods” and “Alcatel Letters,” https://www.birkettlong.co.uk/site/library/commercialclient/public_procurement_lib/procurement_law_jargon_buster .html [https://perma.cc/W8TD-R3XG] (last visited May 14, 2019).
  65. Directive 2007/66 of the European Parliament and of the Council of 11 Dec. 2007 amending Council Directives 89/665/EEC and 92/13/EEC with regard to improving the effectiveness of review procedures concerning the award of public contracts, 2007 O.J. (L 335) 37.
  66. The period is 10 days where the Award decision is notified electronically and 15 days where non-electronic mode of award communication is used. Id.
  67. Id. at 32. The bidder may also seek a range of post-contractual (i.e., post-signing the contract) remedies, such as the remedy of ineffectiveness, fines, or curtailment of the duration of the contract where the court considers that ineffectiveness would not be an appropriate remedy. Id. at 33. Note that consideration of remedies is outside the scope of this article, which focuses on the right to feedback.
  68. See generally Sue Arrowsmith, The Law of Public and Utilities Procurement 798-99, 807 (2d ed. 2005) [hereinafter Arrowsmith].
  69. Directive 2014/24, supra note 11, at 126.
  70. E.g., RPS Consulting Engineers, Ltd. v. Kildare County Council [2016] IEHC 113, 123 (H. Ct.) (Ir.).
  71. E.g., Strabag Benelux NV v. Council of the EU, 2003 E.C.R. II-158.
  72. See Paul Craig & Gráinne de Búrca, EU Law: Text, Cases, and Materials 567 (6th ed. 2015) [hereinafter Craig].
  73. McConnell Archive Storage Ltd. v Belfast City Council, [2008] NICh 3 [12-13] [2008] (N. Ir).
  74. RPS Consulting Engineers, Ltd. v. Kildare County Council [2016] IEHC 113, 132 (H. Ct.) (Ir.).
  75. Id. at 115, 130.
  76. Id. at 123, 130.
  77. Id. This case is analogous to T-59/05, Evropaïki Dynamiki v. Comm’n of Eur. Cmty., 2008 E.C.R. 121, where the EU’s General Court in Luxembourg held that the procurer is under a duty to observe essential procedural requirements. This includes stating the reasons for the rejection of a bid, and so (for example), where scoring alone does not reveal understanding of a net point issue score (e.g., why was a low score given for a competitive price), the reasons behind the low scoring, if not revealed on the face of the award decision, must be given in feedback. Note that the Court of Justice in the subsequent appeal made it clear, however, that once the foregoing is achieved, then disclosure of the full evaluation report is not required. See Case C-476/08 P, Evropaïki v. Comm’n of Eur. Cmty., 2009 E.C.R. I-207.
  78. See Brink’s Security Luxembourg SA v. Commission of the European Communities, 2009 E.C.R. II-03233, 163.
  79. Cf. Arrowsmith, supra note 68, at 799, 801 (discussing the benefits of feedback).
  80. Strabag Benelux NV v. Council of the EU, 2003 E.C.R. II-158.
  81. See id.
  82. E.g., Directive 2014/24, supra note 11, at 126.
  83. Id.
  84. E.g., R v. Higher Educ. Funding Council, ex parte Inst. of Dental Surgery [1994] 1 AII ER 651, 671-72 (Eng.).
  85. E.g., R v. City of London Corp., ex parte Matson [1995] 1 WLR 765, 776 (Eng.).
  86. [1993] 3 WLR 154 (HL) 15 (Eng.).
  87. [2003] AII ER 25 (Eng.).
  88. Mark Elliott, The Duty to Give Reasons and the New Statutory “Makes No Difference” Principle, Pub. L. for Everyone (Apr. 18, 2016), https://publiclawforeveryone.com/2016/04/18/the-duty-to-give-reasons-and-the-new-statutory-makes-no-difference-principle/ [https://perma.cc/88VL-U8YF] (discussing section eighty-four of the Criminal Justice and Courts Act 2015, which inserts new provisions into section thirty-one of the Senior Courts Act).
  89. [2016] IEHC 113, 132 (H. Ct.) (Ir.).
  90. Id.
  91. The Tender Review Service (TRS) was one of the core elements of the project, and the service was nominated as a finalist in the UK Chartered Institute of Procurement & Supply Awards 2013 for Best International Procurement Project, in recognition of its impact on SME suppliers. The TRS research was led and quality controlled by Ceri Evans (one of the authors). See ‘Winning in Tendering,’ supra note 6.
  92. Cahill, supra note 45, at 8, 38.
  93. See Ceri Evans, Public Sector Tendering Challenges for SMEs, Procurer Feedback Provision and Tendering Support Mechanisms: Insights from the Welsh Tender Review Service, in Charting a Course in Public Procurement Innovation and Knowledge Sharing 122–24 (Gian L. Albano et al., eds., 2013).
  94. While this paper analyzes the content in feedback letters provided routinely by procurers to SMEs following a procurement exercise, it is also imperative to acknowledge that feedback can come in various forms, including written feedback, telephone feedback and face-to-face feedback, or a combination of these methods. This study only analyzes the written feedback. It would also have been beneficial to have provided a more detailed breakdown of feedback according to contract size. However, this granular level of detail was unfortunately not readily available for each feedback letter studied. A larger sample would give further weight to the findings; nevertheless, the fact that thirty-nine separate feedback exercises were examined is considered to be statistically significant. The research, of course, also relates to one jurisdiction, the UK, which has been particularly aggressive in adopting numerous SME policies as part of its procurement modernization efforts. However, there is much that other jurisdictions (who are also proactive in this area) can learn from this study.
  95. Feedback letters on file with authors.
  96. Feedback letters on file with authors.
  97. Feedback letters on file with authors.
  98. Feedback letters on file with authors.
  99. Feedback letters on file with authors.
  100. Feedback letters on file with authors.
  101. This is in compliance with E.C.J. case law. E.g., Case C-532/06, Lianakis v. Alexandroupolis, 2008 E.C.R. I-256, 258.
  102. Feedback letters on file with authors.
  103. Feedback letters on file with authors.
  104. Feedback letters on file with authors.
  105. See Supplier Debriefing Guidance, supra note 25.
  106. Feedback letters on file with authors.
  107. Feedback letters on file with authors.
  108. Study information on file with authors.
  109. Feedback letters on file with authors.
  110. Study information on file with authors.
  111. Feedback letters on file with authors. Note that “*” in the table relates to tender feedback only.
  112. Feedback letters on file with authors.
  113. Feedback letters on file with authors.
  114. Feedback letters on file with authors.
  115. Commc’n from the Comm’n, supra note 13, at 10.
  116. Feedback letters on file with authors.
  117. Feedback letters on file with authors.
  118. Feedback letters on file with authors.
  119. Where comparisons were made between the score of the winning and losing bidder, the majority of the score comparisons were made directly with the winning supplier only: in three cases (eleven percent of tender feedback letters), scores were provided for all suppliers or the score range (from lowest to highest) was stated.
  120. Feedback letters on file with authors.
  121. Feedback letters on file with authors.
  122. Feedback letters on file with authors.
  123. Feedback letters on file with authors.
  124. Feedback letters on file with authors.
  125. Feedback letters on file with authors.
  126. Feedback letters on file with authors.
  127. Feedback letters on file with authors.
  128. Feedback letter on file with authors.
  129. This has been made clear in E.C.J case law. E.g., Case T-437/05, Brink’s Sec. Lux. SA v. Comm’n of Eur. Cmty., 2009 E.C.R. II-03233; Case T-183/00, Strabag Benelux NV v. Council of EU, 2003 E.C.R. II-158.
  130. Feedback letters on file with authors.
  131. Feedback letter on file with authors.
  132. Feedback letter on file with authors.
  133. Feedback letter on file with authors.
  134. Feedback letters on file with authors.
  135. Feedback letters on file with authors.
  136. Feedback letters on file with authors.
  137. Feedback letters on file with authors.
  138. While such an approach could be the subject of debate for an entire paper from a legalistic perspective (see supra Part IV) and might only be a workable solution in some incidences, it does serve to demonstrate that options do exist to be innovative and re-engineer the feedback process in such a way that works for both procurers and suppliers.
  139. Feedback letters on file with authors.
  140. Feedback letter on file with authors.
  141. Feedback letter on file with authors.
  142. Feedback letters on file with authors. Note that other examples of good practice were identified in this study, but cannot be revealed because they contain commercially sensitive information.
  143. Feedback letters on file with authors.
  144. Feedback letters on file with authors.
  145. Feedback letters on file with authors.
  146. Feedback letters on file with authors.
  147. See, e.g., Procurement Journey Routes Leadership & Governance: Route 3 – Contract Award - Debriefing, Procurement Journey, https://www.procurementjourney.scot/route-3/route-3-contract-award-debriefing [https://perma.cc/7YT5-U924] (last visited Apr. 21, 2019) (Scot.); Supplier Debriefing Guidance, supra note 25, at 8–9, 14–15. See generally Crown Com. Serv., The Public Contracts Regulations 2015 & For Utilities, The Utilities Contracts Regulations 2016: Guidance on the Standstill Period 3–5 (2016) (Eng.).
  148. See generally Off. of the Procurement Ombudsman, Procurement Practices Review – Chapter 2: Supplier Debriefings (2009). This document (in a similar vein to this study) singles out the importance of standardized feedback provision. Id. at 11.
  149. See, e.g., Glover, supra note 47, at 10.
  150. GHK, supra note 23, at 59.
  151. Both the Wales Procurement Policy Statement and Circular 10/14 (the Irish Government policy on facilitating SME involvement in public procurement) were examined as illustrative examples, as was the latest European Commission’s procurement publication, Making Public Procurement Work in and for Europe (which sets out the overall public procurement policy framework for the EU). Commc’n from the Comm’n, supra note 13, at 2.
  152. Here, legal compliance is determined on the basis of the need for a public body to explain the decision to award a contract to one bidder over another.
  153. Cahill, supra note 45, at 50–52 (listing the recommendations).
  154. See generally id. at 51.
  155. Fed’n of Small Bus., supra note 49, at 5–6.
  156. Cent. Gov’t Direct and Indirect Spend, supra note 14, at 1.
  157. The standardized feedback methodology is designed to align with European Commission objectives to standardize public procurement forms and procedures as much as practicable, thus standardizing divergent practices across the EU; for example, as has happened with the European Single Procurement Document (ESPD). For an explanation of the ESPD and reference to standardization, see Commission Implementing Regulation 2016/7 of 5 Jan. 2016 establishing the standard form for the European Single Procurement Document, 2016 O.J. (L 3) 17. The standardized feedback methodology will also encourage SMEs to bid for cross-border contracts, thus meeting European Commission policy goals that make reference to the importance of opening up access for SMEs to cross-border contracts. Commc’n from the Comm’n, supra note 13, at 9–10.
  158. See, e.g., Glover et al., supra note 47, at 56–57, 59; see also supra Part VII(A).
  159. Study information on file with authors.
  160. See, e.g., Directive 2014/24, supra note 11, at 126.
  161. See Commc’n from the Comm’n, supra note 13, at 10–11. This publication sets out the overall public procurement policy framework for the EU and under the policy strand “Improving access to procurement markets” states: “[a]nother way to improve access is to build trust among market players, including SMEs, to participate in procurement procedures.” Id. at 9–10.
  162. Feedback letters on file with authors.
  163. Feedback letters on file with authors.
  164. Commc’n from the Comm’n, supra note 13, at 10–11.
  165. E.g., Directive 2014/24, supra note 11, at 126.
  166. See, e.g., Case T-437/05, Brink’s Sec. Lux. SA v. Comm’n of Eur. Cmty., 2009 E.C.R. II-03233; Case T-59/05, Evropaïki Dinamiki v. Comm’n of Eur. Cmty., 2008 E.C.R. II-157; Case T-183/00, Strabag NV Benelux v. Council of EU, 2003 E.C.R. II-158; C-81/98 Alcatel Austria AG and Others v. Bundesministerium für Wissenscahft un Verkehr, 1999 E.C.R I-7708-09, I-7711-12.
  167. See, e.g., Alison Jones & Brenda Sufrin, EU Competition Law: Text, Cases, and Materials 892 (6th ed. 2016); Stephen Weatherill, Cases & Materials on EU Law 14, 16–18, 61, 261, 649 (12th ed. 2016); Craig, supra note 72; John Fairhurst, Law of the European Union 658 (9th ed. 2012).
  168. European Code of Best Practices, supra note 44.
  169. See, e.g., Strabag, 2003 E.C.R. II-158.
  170. See, e.g., Anthony Flynn & Paul Davis, Investigating the Effect of Tendering Capabilities on SME Activity and Performance in Public Conract Competitions, 35 Int’l Small Bus. J. 449, 449, 454, 465 (2016) (referring to SME barriers relating to basic tendering skills) [hereinafter Flynn & Davis]; Katri Karjalainen & Katariina Kemppainen, The Involvement of Small- and Medium-Sized Enterprises in Public Procurement: Impact of Resource Perceptions, Electronic Systems and Enterprise Size, 14 J. Purchasing and Supply Mgmt. 230, 232 (2008) [hereinafter Karjalainen]; Kim Loader, Supporting SMEs Through Government Purchasing Activity, 6(1) Int’l J. Entrepreneurship & Innovation 17, 18, 21 (2005) (referring to SME barriers relating to lack of knowledge of the tendering process) [hereinafter Loader].
  171. See, e.g., Case T-437/05, Brink’s Sec. Lux. SA v. Comm’n of Eur. Cmty., 2009 E.C.R. II-03233.
  172. See generally R v. Sec’y of State, ex parte Doody [1993] 3 WLR 154 (HL) 23 (Eng.) (discussing requirement in the criminal context that the government must provide an explanation for its decisions).
  173. See, e.g., Commc’n from the Comm’n, supra note 13, at 10.
  174. Id. at 9–10.
  175. Flynn, supra note 35, at 108.
  176. Feedback letters on file with authors.
  177. See Commc’n from the Comm’n, supra note 13, at 10–11.
  178. Directive 2014/24, supra note 11, at 126; see also Case C-532/06, Lianakis v. Alexandroupolis, 2008 E.C.R. I-256, 257.
  179. See, e.g., Commc’n from the Comm’n, supra note 13, at 8, vis trust-building measures to encourage SME public procurement participation.
  180. See EU and national case law considered supra, Part IV.
  181. Commc’n from the Comm’n, supra note 13, at 10.
  182. See, e.g., Strabag NV Benelux v. Council of EU, 2003 E.C.R. II-158.
  183. European Code of Best Practices, supra note 44.
  184. E.g., Case T-437/05, Brink’s Sec. Lux. SA v. Comm’n of Eur. Cmty., 2009 E.C.R. II-03233.
  185. See generally R v. Sec’y of State, ex parte Doody [1993] 3 WLR 154 (HL) 23 (Eng.) (discussing requirement in the criminal law context that the government must provide an explanation for its decisions).
  186. See, e.g., Flynn & Davis, supra note 170; Karjalainen, supra note 170; Loader, supra note 170.
  187. See Commc’n from the Comm’n, supra note 13, at 10.
  188. See id. at 9–10.
  189. See id. at 10.
  190. Feedback letters on file with authors.
  191. See supra Part IV, where the article demonstrates that it is possible in some cases to adopt such an approach and for it to be legally sound.
  192. While no universal definition of Supplier Development in the context of public procurement exists, it would typically refer to business support (e.g., tender training, tendering information, and tender reviews) provided to SMEs to help them become tender-ready (or contract-ready). See The World Bank, International Experience in Supplier Development 2–4 (2008), available at http://documents.worldbank.org/curated/en/355201468273033230/International-experience-in-supplier-development [https://perma.cc/6XXM-3QM4].
  193. See Nicola Dimitri et al., Handbook of Procurement 55–56 (Nicola Dimitri et al. eds., 2006).
  194. Preuss, supra note 38, at 791–92, 803.
  195. Interestingly, the European Commission’s European Code of Best Practices Facilitating Access by SMEs to Public Procurement Contracts states that the stakeholders consulted during the preparation of the Best Practices document stressed that the key thing to facilitate SME access to public procurement is not legislative change, but rather change in relation to public bodies’ procurement culture. European Code of Best Practices, supra note 44, at 13.
  196. Feedback letters on file with authors.
  197. Feedback letters on file with authors.
  198. Feedback letters on file with authors.
  199. See Policy-Practice Divide, supra note 61, at 559.
  200. See Commc’n from the Comm’n, supra note 13, at 10.