Making public procurement better and fairer with supplier feedback

18 Dec 2018 01:51 PM

Blog posted by: Nida Broughton, Jason Waterman and Hubert Wu, 18 December 2018 – Categories: Improved outcomes.

Feedback between buyers and sellers can make markets fairer and more efficient. Saying thanks to a shopkeeper, writing a review on Amazon, and rating a hotel on TripAdvisor are a few examples of this. These activities can reward top performers, better match supply and demand, and hold suppliers and buyers accountable for their actions.

Rating and feedback systems have large potential benefits for consumers. They can also benefit small businesses. With more information on quality and past performance, purchasers may be more likely to give a high-performing small business a chance, rather than automatically opting for a larger supplier with brand recognition. We’ve seen this in the restaurant industry, where an analysis of restaurant ratings on Yelp found that star ratings were effective in driving demand to independent businesses over chains that diners were more aware of.

A team within the Behavioural Insights Team (BIT) worked with the Crown Commercial Service (CCS) to pilot the collection (but not the display) of simple feedback from public sector buyers on their purchases of IT and office supplies. The pilot’s results surprised and encouraged us.

What we did

The pilot ran for a year and took place on the CCS online test platform for its Tech Products 2 and Office Supplies catalogues. Public sector buyers can use this platform to buy a wide range of products. During the pilot, we:

• collected feedback on suppliers from buyers by having them answer some questions (see below)
• analysed response rates, feedback content, and whether there was a difference in the performance of small and larger businesses
• returned buyer feedback to the relevant supplier

What we asked buyers

• Were there any delays in the product being delivered?
  • If yes, by how many days was the product delayed?
  • If yes, did the supplier communicate with you?
• Was the order delivered in full?
• Overall, how satisfied are you with the supplier’s performance? [Star rating out of five]

We collected around 1,000 individual responses to these questions during the pilot.

What we found

Four key results came from our pilot.

• First, many buyers used our system

The overall response rate was 10.8%. This may seem low, but is actually much larger than the rates of between 2% and 5% seen on major business-to-consumer platforms. We believe this rate could be increased with further iteration and testing of behaviourally informed messages and prompts.

• Second, the majority of suppliers found feedback useful, wanted it to continue, and used it to improve their performance

More than three-quarters of suppliers said they found the performance feedback useful and would like to continue receiving it. Suppliers also responded to feedback they received: when we returned the results to them, their subsequent performance improved across all the metrics we measured. Although this pilot was small, we think it demonstrates the potential for feedback to improve how public services are delivered and improve value for money for the taxpayer.

• Third, buyers’ overall star ratings closely tracked their responses to more objective questions (for example, whether the product was delayed)

This consistency suggests that the overall reviews are likely to be relatively objective and representative of diverse performance criteria.

A general challenge that feedback platforms have to overcome is to ensure that feedback is honest, reliable and comes from a wide range of customers (with both positive and negative experiences). It is important to make sure such issues are monitored and addressed, especially when using feedback in a public sector context.

• Our final finding is the most intriguing: smaller suppliers performed slightly better than larger businesses (see bar chart below)

Smaller suppliers were more likely to receive four and five star ratings, and less likely to receive one star ratings than larger businesses. This difference was statistically significant.

More work is needed to understand the potential reasons for this difference in performance, and whether we would see similar patterns in a larger pilot. However, it suggests that introducing feedback into public procurement could help demonstrate the high-quality services on offer from small, as well as large, suppliers. These tools could play an important role in helping the Government achieve its target for a third of procurement spending to be with SMEs by 2022.

Next steps

CCS and BIT are exploring how to build upon the initial trial. If you’d like to learn more about the pilot or our work in this area, please do get in touch.

There are many details to work out about what a usable feedback system should look like and how it would work, including with respect to procurement regulations that ensure public procurement is fair and open. But we believe that there is real potential for feedback to make public procurement better and fairer for all in ways that don’t exist today.

Why we need better data to improve the lives of ageing populations