HEPI-The past, present and future of research assessment

The Higher Education Policy Institute (www.hepi.ac.uk) has published a new collection of pieces entitled Research Evaluation: Past, present and future (HEPI Report 152), edited by Dr Laura Brassington. The dozen different authors consider the origins of UK research evaluation, the outcome of the latest Research Excellence Framework (REF 2021) and options for the future of research assessment.

Dr Laura Brassington, HEPI’s Policy Manager and the Editor of the collection, said:

‘In its relatively short history, the REF has engendered considerable controversy. Our edited collection of pieces by academics, policymakers and other higher education experts provides a range of views on the pros and cons of this controversial topic, which stands to shape the future of the UK’s global position in research and development.’

In his chapter, Nick Hillman, HEPI Director, argues the inextricable link between eligibility for research funding from the REF and the undergraduate fee cap of £9,250 should be reconsidered. He said:

‘It is embarrassing that the UK, as a research superpower that generally punches above its weight, has not had a Minister for Science at Westminster for two months. If we are to get anywhere near the Conservatives’ manifesto commitment to spend 2.4% of GDP on research and development, it is essential the new Prime Minister appoints a big figure to this role in the next few days and that they are given the space to make a difference.

‘World-leading research rests on many things, including secure long-term funding, a sufficient number of researchers and a good research environment. These are the sorts of things that are reflected in the REF exercise, the last round of which has recently finished. The rules are now being reviewed for the future.

‘I hope this collection of essays will be at the top of the in-tray for any new Minister for Science because it provides a range of views on how best to judge and fund the fantastic research found up and down the UK. With the proper support in place, the solutions to the world’s grand challenges are likely to emerge, at least in part, from our fantastic universities.’

In his Foreword to the report, Bahram Bekhradnia, who presided over several Research Assessment Exercises as the Director of Policy for the Higher Education Funding Council for England (Hefce), writes:

‘[T]he REF, like the RAE [Research Assessment Exercise] before it, has evolved – and there can be little doubt that it has been one of the reasons for the international pre-eminence of UK universities in research. But the REF has downsides, and arguably these have become so serious as to raise questions about whether it can continue in its present form. As a dominant feature of academic life, it has led academics – and university leaders – to focus on research over teaching and other academic activity. The measures taken to increase the value placed on teaching have been puny and unsuccessful compared to the imperative the REF provides to focus on research. …

‘A fundamental rethink is required of the REF and indeed more generally of research policy and funding – a rethink that is already under way with the Future Research Assessment Programme (FRAP) review – and the series of essays contained in this report will make a welcome contribution to this rethink.’

In the concluding chapter, ‘Passing on the baton’, David Sweeney CBE, the recently departed Executive Chair of Research England, writes:

‘[I]t is a significant challenge to capture sufficient rigour and richness in the assessment of research environment and, as I perceive it, this is the key issue in considering future research assessment. The move from theorising about research culture to delivery mechanisms is challenging and it is simply insufficient to say, as one change proponent was quoted recently, that “there must be a way of doing it”. That is the last refuge of an official trying to rescue a failing policy implementation and it is a pitfall which the academy must confront and address. And while I have talked about “the academy” here there also needs to be discussion about who is inside that academy. Our research teams now include experts with a range of professional responsibilities, whether they be technicians, statisticians, librarians, research managers and so on. In considering research assessment the work of those teams should be assessed appropriately, not just the outputs which bear the names of the principal investigators and some colleagues. These discussions are less well-developed in the theory and even further from a delivery mechanism.’

In his chapter on how the REF works, Nick Ellison, Professor of Social Policy at the University of York, writes:

‘Does REF skew the kinds of research undertaken by UK higher education institutions? Does it skew the content and character of academic outputs? Do larger Units of Assessment with significant economies of scale inevitably do better? There are definitely debates to be had here but, on the upside, with its efforts to attend to quality and not just quantity, and the enormous care taken by sub-panels to arrive at fair judgements, REF may well be the “least worst” option, assuming, that is, that research assessment of some kind is required at all.’

In her chapter looking at the history of research assessment, Dr Helen Carasso from the Education Department at the University of Oxford writes:

‘Although not always as immediately evident, the perceived tension between “excellence” and diversification that underlies many of the policy debates about teaching within higher education institutions is also at the core of discussions about the nature of the UK’s academic research base. …

‘The legacy of the six RAEs was therefore a boost for the international standing of UK research, through concentrated support for certain disciplines and universities, at the cost of increased stratification of institutional research capability and individual research opportunities within the country.’

In her chapter on the quality of the research base in London, Dr Diana Beech, the Chief Executive Officer of London Higher, writes:

‘While the very nature of the REF exercise dictates that institutions are judged on their research excellence individually, research in the capital is very rarely conducted by institutions in isolation, and the benefits of this “world-leading” research are usually felt far outside London. As the REF 2021 Impact Cases Studies confirm, money spent on research in London brings benefits to populations across the regions as well as across the globe.’

In her chapter on the evolution of research assessment in Wales, Professor Cara Aitchison, President and Vice-Chancellor of Cardiff Metropolitan University, writes:

‘In a small nation, where the leaders of Wales’ eight universities can readily fit around a dinner table, it is relatively easy to forge both policy and partnerships between universities. Moreover, the proximity of universities and government, in a nation that still considers universities to be squarely within the public sector, lends a particular level of accountability to “impact” that is perhaps less visible to or manageable by government at Westminster. Indeed, much of the research and innovation landscape of Wales exemplifies the ‘Triple Helix’ model of the intertwining of government, industry and education in developing and delivering research-led solutions to entrenched social and economic problems, including Wales’ long-standing challenge of low Gross Value Added (GVA) relative to the rest of the UK.’

In her chapter on research culture, Clare Viney, CEO of the Careers Research & Advisory Centre Ltd (CRAC), writes:

‘The way research is being conducted is changing. Increasingly, researchers are expected to work collaboratively, interdisciplinarily, inter-sectorally and internationally. They are expected to share their research, data and publications openly, and demonstrate the social and / or economic impact of their research. These developments raise the importance of values such as research integrity, ethics and reproducibility. …

‘In future research assessment, Vitae would like to see further emphasis on researchers and researcher development, and research culture within the environment assessment.’

In their chapter on funding, Ellie Russell and Jennie Eldridge of the University Alliance, write:

‘There is a lot to celebrate about the wide distribution of research excellence and the diverse ways this research is benefiting society that has been demonstrated in the REF 2021 results. The increase in the volume of high-quality research across the UK is also a reminder of what is at stake in the political and funding uncertainties surrounding universities and research collaboration. …

‘To meet the local and global challenges we face, universities will need to maintain and increase their RDI [research, development and innovation] capabilities, but this must be underpinned by a sustainable, long-term funding settlement. The longevity and scale of the higher education sector does not mean it can simply absorb the impact of real term cuts to the unit of resource and do more with less in perpetuity.’

In his chapter, Professor Peter Mandler, Professor of Modern Cultural History at Gonville and Caius College, Cambridge, writes:

‘The time is ripe for a root-and-branch reconsideration. Rip up the rulebook and start again. Think seriously about whether, as is often suggested, a simple headcount might lead to rough justice without the thousands of pages of boilerplate and the hundreds of meetings and exercises. And get back to basics. Anything worthy of the name “research excellence” has to put excellent research, not a lengthening government or managerial wish-list, at its heart.’

In his chapter, Professor James Wilsdon, Director of the Research on Research Institute, writes:

‘Before hurtling into the next REF, the question we need to answer is not: “are metrics a better way of assessing research?” Instead, as others have argued, we need to start by revisiting and clarifying what the REF is for. Its stated purposes have multiplied in recent years. … Can a single assessment framework deliver on six, seven or eight simultaneous objectives? What hierarchy of importance should be applied to these? What happens when they are in tension?

‘These are some of the fundamental questions that the FRAP [Future Research Assessment Programme] process needs to answer. Then, and only then, can we have a sensible discussion about design options, and the role of metrics as a methodology to deliver them.’