WIREDGOV

The report examines whether to inform decisions across the BBC, the Corporation efficiently, effectively and economically understands how people use and respond to the full range of its services.

Downloads

• Full Report (pdf - 511KB)
• Summary (pdf - 92KB)
• BBC interactive data visualisation ( - 1KB)
• Press Release

Building on the effective audience measures that exist for TV and radio, the BBC has rightly sought to address areas where data does not currently meet its needs, particularly in measuring and understanding online audiences. However, there have been delays to implementing both mandatory sign-in for iPlayer and Cross-Media Measurement, and access to third-party data is not fully within the BBC's control. Until the BBC achieves the benefits it intends from these initiatives, the Audiences team's ability to continue providing value for money is at risk.”

Amyas Morse, head of the National Audit Office, 6 December 2017

The BBC’s Audiences team has supported the BBC effectively based on its extensive and robust data for broadcast TV and radio. However, the BBC’s data for online services are not always sufficient for its needs. In response to changes in technology and audience behaviour, the BBC is increasingly analysing signed-in user data and improving its data on how people use the full range of BBC services. According to a report yesterday by the National Audit Office, both these initiatives have the potential to bring benefits to the BBC but have faced delays in implementation.

Audience data and insights are critical to the success of the BBC’s public service broadcasting, affecting decisions the Corporation takes about how to spend the majority of its £3.8 billion of licence fee income. These data and insights are vital for performance monitoring, informing the creation of new content and services, and helping shape the BBC’s strategy.

The BBC’s Audiences team is responsible for collecting and disseminating audience data and insights, and spent £22.4 million in 2016-17, with costs split between research and measurement (£16.6 million) and staff (£5.8 million).  It monitors audience performance, including figures for TV viewing, radio listening, and the use of online services, as well as audience appreciation and perception data. It also carries out bespoke research, including on audience behaviour trends and how particular demographic groups, such as people of different ages and ethnicities, use the BBC. Audience responses to BBC content vary, as do the amount and range of services that different people use. For example, in 2016-17, all individuals aged 16 and over spent, on average, around 17 hours per week using BBC services, compared to around 11 hours for younger (16- to 34-year-old) audiences.

The BBC’s audience measurement needs are changing because of significant changes in the media environment, including new technology and competition, and shifting audience behaviour. The BBC’s online services were visited by 40 million unique UK-based visitors in June 2017, up 36% since June 2014. The use of the BBC’s iPlayer increased by 151% (from 1.5 billion to 3.8 billion programmes accessed per year) between 2010 and 2016. And elsewhere, the number of UK households with subscription video on-demand services, such as Netflix, rose by 145% (from 3.8 million to 9.4 million) between 2014 and 2017. These changes are particularly pronounced among younger people.

While the BBC already collects data about all its online services from various internal and external sources, these do not always meet its needs, and can be difficult to align and reconcile. This means that the BBC has faced measurement challenges where services exist entirely online, such as BBC Three. In addition, where BBC content is designed to be accessed through third-party online services, such as Facebook, Twitter and YouTube, the BBC is reliant on these organisations to supply audience data. The BBC has no direct control over this data’s design or quality, or what data are shared.

The BBC is responding to these issues through two significant initiatives – the development of the BBC-ID and Cross-Media Measurement. The BBC is increasingly asking and requiring people to sign-in with a BBC-ID to access content. This is enabling the BBC to enhance its understanding of users’ behaviour across its website, mobile applications and the iPlayer, facilitating the development of more personalised services.

Mandatory sign-in for iPlayer on web and mobile applications, whereby people must use their BBC-ID to access iPlayer content, began in June 2017, 15 months later than originally planned, owing to technical challenges and the BBC’s decision to collect more demographic data from users. This delayed the BBC in gaining access to, and starting to analyse, data about a large number of signed-in users. However, by the end of October 2017, the BBC had 12.6 million active monthly signed-in users, and had started to analyse and report on the new BBC-ID data that users are generating.

The BBC’s Cross-Media Measurement project is also delayed. The project aims: to improve measurement of audience perceptions; to provide better data on people’s use of, and time spent with, all BBC services; and to introduce new ways of integrating high-volume datasets to give further insight. The project has introduced a new daily diary-based survey and a pilot version of a new children’s cross-media measurement approach. However, it has also encountered difficulties recruiting research participants and implementing new metering technology, causing delays of at least eight months to important aspects of the project’s scope. The delays have led the BBC to deduct £25,000 from payments (out of £1.9 million expected contract costs in 2017-18) to the contractor, Ipsos MORI. These deductions are equivalent to the annual limit for financial penalties, meaning that, unless it agrees new arrangements with the contractor, the BBC will not be able to levy further financial penalties until April 2018. In October 2017, both parties began discussions to work towards new contractual terms and payment arrangements.

The extent to which the Audiences team can provide effective support to decision-making in the future will depend on the successful implementation of BBC-ID analytics and Cross-Media Measurement. The increasing range of data available to the BBC also means the Audiences team faces the growing challenge of communicating complex insights in ways that content creators, commissioners and schedulers can easily engage with.

Overall, the Audiences team expects to spend £23.7 million in 2017-18, an 11% real-terms increase since 2014-15, reflecting investment in digital analytics skills needs and measurement and analytics projects. The BBC has identified a need for 37 extra analysts, to focus on real-time and advanced analysis of BBC-ID data, but has noted that these skills are in high demand.

The NAO is recommending that the BBC should review how it is measuring and tracking all of the intended benefits of BBC-ID analytics and Cross-Media Measurement, and the timeliness with which these benefits are being achieved. Given the business need for, and importance of, Cross-Media Measurement, the NAO says that the BBC should consider including this project in its critical projects portfolio, particularly in light of the delays to date and the ongoing project risks.