WIREDGOV

The 2015 independent report ‘The Metric Tide’ [Note 1] highlighted growing pressure on higher education institutions, researchers, funders and policymakers to use metrics in managing and assessing research. Metrics form part of an evolving and increasingly digital research environment, where data and analysis are playing an ever greater role. However the current description, production and use of these metrics are at best experimental and open to misunderstanding, and can lead to negative effects and behaviours as well as positive ones.

A new Forum for Responsible Metrics is being set up as a partnership between HEFCE, Research Councils UK, Wellcome, Universities UK and Jisc to advance the agenda set out in ‘The Metric Tide’.

The forum will develop a programme of activities to support the responsible use of research metrics in higher education institutions and across the research community in the UK. This will include advice on, and work to improve, the data infrastructure that underpins metric use. In this way, the whole research community can benefit from the more judicious use of metrics.

 Responsible metrics can be understood in terms of the following dimensions:

• Robustness – basing metrics on the best possible data in terms of accuracy and scope
• Humility – recognising that quantitative evaluation should support, but not supplant, qualitative, expert assessment
• Transparency – keeping data collection and analytical processes open and transparent, so that those being evaluated can test and verify the results
• Diversity – accounting for variation by field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system
• Reflexivity – recognising and anticipating the systemic and potential effects of indicators, and updating them in response.

In addition, the forum will offer advice to the UK higher education funding bodies on how quantitative indicators might be used in assessing research outputs and environments, as part of the funding bodies’ consultation on arrangements for the next Research Excellence Framework (REF) exercise.

Professor David Price, Vice-Provost (Research) at University College London (UCL), will chair the Forum. Professor Price said:

‘From my work at UCL and as a REF sub-panel chair, I am keenly aware of both the utility and power of metrics used wisely, but also of the great hazards if they are abused or applied without the necessary understanding of their limitations. I know there is a great deal of insight across the sector on how we can make responsible use of metrics, sensitive to their sources and context, and I am determined that the forum will make a decisive contribution in accelerating the adoption of good practices in all UK research organisations.’

The forum’s membership draws on representatives and experts from university management, academic leadership, research administration and research metrics. The five partners recognise the wide interest in this work and expect to contact and involve other stakeholders in discussions over the coming months. 

Notes

• ‘The Metric Tide: Report of the independent review of the role of metrics in research assessment and management’ was published in July 2015. It drew together the findings and recommendations of an independent review of the role of metrics in the assessment and management of research. The review was chaired by Professor James Wilsdon, then at the Science Policy Research Unit at the University of Sussex, now Professor of Research Policy and Director of Impact and Engagement at the University of Sheffield. The report can be found on our website at www.hefce.ac.uk/rsrch/metrics/.

• The following are members of the forum:
  • Professor David Price, Vice-Provost (Research), University College London (Chair)
  • Steven Hill, Head of Research and Health Policy, HEFCE
  • Ian Viney, Director of Strategic Evaluation and Impact, Medical Research Council (representing Research Councils UK)
  • Chonnettia Jones, Head of Insight and Analysis, Strategy, Wellcome
  • Jamie Arrowsmith, Programme Manager (Research Policy, Efficiency and Effectiveness), Universities UK
  • Rachel Bruce, Deputy Chief Innovation Officer, Jisc
  • Stephanie Bales, Director of Research and Business Services, Northumbria University (representing the Association of Research Managers and Administrators)
  • Professor Roger Kain, Dean and Chief Executive, School of Advanced Study, University of London (representing the British Academy)
  • Professor Linda King, Pro Vice-Chancellor, Research and Global Partnerships, Oxford Brookes University
  • Professor Ottoline Leyser, Director, Sainsbury Laboratory, University of Cambridge (representing the Royal Society)
  • Keith McDonald, Assistant Director, Research and Innovation, Scottish Funding Council (representing the UK higher education funding bodies outside England – the Scottish Funding Council, the Higher Education Funding Council for Wales, and the Department for the Economy Northern Ireland)
  • Glenn Swafford, Director of Research Services, University of Oxford
  • Professor Mike Thelwall, Professor of Information Science, University of Wolverhampton
  • Professor James Wilsdon, Professor of Research Policy and Director of Impact and Engagement, University of Sheffield
  • Professor Lesley Yellowlees, Professor of Inorganic Electrochemistry, University of Edinburgh
  • Ben Johnson, Research Policy Advisor, HEFCE (secretary).
  • The forum is being established as a partnership between HEFCE, Research Councils UK, Wellcome, Universities UK and Jisc.

• ​​The partners in the Forum are:
  • The Higher Education Funding Council for England (HEFCE): www.hefce.ac.uk  
  • Wellcome: https://wellcome.ac.uk/  
  • Universities UK: www.universitiesuk.ac.uk  
  • jisc: https://www.jisc.ac.uk/  
  • Research Councils UK: www.rcuk.ac.uk.

• The seven UK Research Councils are:
  • Arts and Humanities Research Council: www.ahrc.ac.uk
  • Biotechnology and Biological Sciences Research Council: www.bbsrc.ac.uk
  • Economic and Social Research Council: www.esrc.ac.uk
  • Engineering and Physical Sciences Research Council: www.epsrc.ac.uk
  • Medical Research Council: www.mrc.ac.uk
  • Natural Environment Research Council: www.nerc.ac.uk
  • Science and Technology Facilities Council: www.stfc.ac.uk.