WIREDGOV

Research published today by the Higher Education Funding Council for England (HEFCE) analyses league tables that provide rankings of universities for newspapers and on the web. It also looks at how universities and colleges in England respond to such rankings. While there is clearly a demand for league tables among prospective students and others, many feel dissatisfied with the way they are compiled.

The research analysed five university league tables and investigated how higher education institutions (HEIs) respond to them, including their effect on university managers, governors and their decision-making. It is based on a survey and case studies.

The researchers found that the rankings of the most influential league tables of UK institutions largely reflect reputational factors such as entry qualifications, level of degree attained and Research Assessment Exercise grades. Their analysis indicates that:

• measures used in the tables are largely determined by the data available, not by clear definitions of quality
• some of these measures are poor indicators of the qualities identified
• methods for calculating scores are not always transparent, and some produce non-standardised results.

The report finds that despite these imperfections, institutions are strongly influenced by league tables - although they are reluctant to acknowledge this. Many even go so far as to use the tables as key performance indicators. Another finding is that institutions respond to the results of the National Student Survey (NSS): there is widespread evidence of institutional actions and initiatives arising from NSS results.

Universities and colleges are concerned that efforts to move up existing league tables may conflict with government priorities on widening participation: for example, the proportion of good degrees attained might be adversely affected by accepting a high proportion of students with non-standard qualifications. They also see possible tensions between league table performance and academic standards, community engagement and provision of socially valued subjects (for example, in health-related areas).

There is evidence that league tables influence student choice, particularly for 'traditional' prospective students, although subject and location are still the top factors when selecting a degree programme. League tables are increasingly used by international students applying to study in the UK, and by international academics. It is widely believed that league tables will become more influential if higher education becomes more competitive and the cap on fees is raised. Professor David Eastwood, Chief Executive of HEFCE, said:

'There clearly is a demand for league tables, and they draw attention to important issues such as the "student experience", retention and employability. However, there are also questions about league tables' quality, impact and their possible perverse incentives. It is important that the information used by prospective students to guide their choice of subject and institution is both informative and accurate. The league tables also have a much wider impact - for example, on institutions' reputations and potentially on the behaviour of academics, businesses, and potential benefactors. Governing bodies take an interest in them as a means of measuring institutional performance, sometimes seizing on them in default of other, more appropriate, indicators of institutional performance.

'Our purpose in commissioning this research was not to create an authorised "official" league table or to endorse any particular approach, but to stimulate informed debate about league tables across the HE sector. We hope that this will lead to improvements in league table methodologies and enable users to understand their complexities better and avoid misunderstandings. We also want to help higher education institutions to develop approaches that satisfy the legitimate information needs of their stakeholders.'

The study was commissioned by HEFCE from the Centre for Higher Education Research and Information (CHERI) at the Open University and Hobsons Research.

Notes to Editors

1.   The report, 'Counting what is measured or measuring what counts? League tables and their impact on HEIs in England' (HEFCE 2008/14) is available.

2.   The study examined five league tables:

• three national tables:
  • Sunday Times Good University Guide
  • The Times Good University Guide
  • The Guardian University Guide
• two world rankings:
  • Academic Ranking of World Universities published by Shanghai Jiao Tong University Institute of Higher Education
  • Times Higher Education Supplement (now THE)/Quacquarelli Symonds (THES-QS).

3.   The research team analysed the methods and results of the five league tables. This analysis was supplemented by a 2007 online survey of the views of HEIs in England about league tables to which 91 institutions responded. The researchers also undertook six case studies of league tables' impact on institutions.