In this article Tony Roche discusses a movement to ensure equitable research evaluation, highlighting the Hidden REF initiative. This campaign, starting in 2019, aims to recognize all research contributions, including roles often overlooked in traditional assessments focused on publications. The Hidden REF Competition showcased diverse research outputs, highlighting non-traditional contributions from roles like librarians and software engineers. Despite their value, such outputs have been underrepresented in evaluations like the UK’s Research Excellence Framework (REF). The 2023 Festival of Hidden REF and its resulting white paper advocate for broader recognition of non-traditional research in future evaluations. This shift seeks to retain talent in academia by creating a more inclusive research environment. The initiative also aligns with efforts by organizations like the Responsible Research in Business and Management network and Emerald Publishing to foster impactful, societally beneficial research. The Hidden REF aims to drive progressive changes in research culture and evaluation.
Current approaches to the assessment of research impact overlook many of the people who are vital to the success of research itself. Accelerated by technological advancements including the increasingly widespread utilisation of artificial intelligence, research is changing across all fields, including the span of disciplines explored within the business school environment. Today, we rely on many people and many different skills to conduct research, but the approach to recognising research excellence remains fixated on scholarly publications, primarily journal articles, conference proceedings and books.
Founded in 2019, The Hidden REF started as a short campaign to promote all research outputs and every role that makes research possible, focusing on the UK’s Research Excellence Framework (REF)1. The campaign culminated with a Competition2 which showcased the diversity of UK research outputs, and the incredible people who work in largely hidden roles, from librarians to software engineers, research administrators and managers, all of whom make vital but largely unrecognised contributions to research.
Rigorous, peer-reviewed journal articles, conference proceedings and books are vital, of course, but publications are not the only valuable output from research. Faculty in business schools are fundamentally and inextricably reliant on a broad range of work and a diverse group of people, yet recognition for the success of the team effort is directed to the exclusive group who are named as authors of the publications. The prevailing culture damages our ability to conduct research by not recognising many of the people who are vital to research itself.
Focusing on the UK, the proportion of research outputs that are not related to publications or books fell from 6% in the 2008 Research Assessment Exercise, to 2.4% in REF 2021. With so much funding and status for UK Business Schools dependent on the REF results, institutions have followed an ever more conservative approach to the exercise. They have focussed almost exclusively on something that, they believe, is the only output that is understood and valued in research. Yet data released by the REF suggests that ‘non-traditional’ outputs perform just as well as traditional publications in terms of the proportion awarded a 4* rating and the associated funding that confers.
The focus on publications and books in the REF is symptomatic of a research culture that overlooks many of the people who are vital to research. If we steer this towards a culture that recognises everyone involved in the research process, we will create an environment that is both more equitable and more able to advance research by ensuring that the best talent is retained in academia.
To explore how wider contributions to research might be brought to life in research assessment, 2021’s Hidden REF Competition generated 120 entries from more than 60 universities. A third of those submissions were made in the ‘Hidden Role’ category. Submissions were diverse, including for example: a proposal concerning 18 street children from Ghana, the Democratic Republic of Congo, and Zimbabwe who had described their experiences of living on the streets; a crowd-sourcing platform for increasing access to collections at the British Library; and an open-source project to make data science reproducible, ethical, collaborative, and inclusive to the benefit of all.
The engagement with the competition and the attention this generated across the research sector created momentum for continuing the campaign for developing a more equitable research environment. As a result, plans were then drawn up to hold the Festival of Hidden REF in September 2023.
The Festival brought together a range of researchers and other research professionals who support research more broadly alongside policymakers, publishers and others involved in research assessment. Over the course of the one-day event, delegates focused on ways to further raise recognition for the diversity of roles and practices that contribute to research, working towards The Hidden REF’s overarching objective of a fairer system of evaluating success in research. The principal output of the Festival is a white paper3. Organised around two main themes, the white paper sets aside journal articles, conference proceedings and book publications and instead defines and evaluates ‘non-traditional’ research, promoting its inclusion in the UK’s next Research Excellence Framework (REF 2029).
Given the ambition of all of this work and the desire to broaden research evaluation beyond journal articles, conference proceedings and book outputs, one might ask whether publishers have a place in co-creating the future of research evaluation.
The white paper also sets out the structure of the second Hidden REF Competition4; any person, team, department or business school as an institution seeking to raise awareness of a research output (that is not a journal article, conference proceedings, or book!), is warmly encouraged to contribute to the framing of the competition categories and submit an entry prior to the deadline in September 2024. The forthcoming competition also presents an opportunity to experiment with evaluation approaches that would not be possible in the mainstream REF 2029. As such, Hidden REF 2024 is uniquely well-placed to assess what works in the evaluation of ‘non-traditional’ outputs from business schools and other institutions. If these conclusions are implemented in REF 2029, sectoral confidence in the fair and robust evaluation of ‘non-traditional’ research outputs will grow, and Higher Education Institutions (HEIs) will include more non-traditional outputs in their REF 2029 output submissions. This is at the heart of the movement to drive more progressive approaches to research evaluation, made a reality by changes in the culture of the research sector itself.
Honing in again on the business school community, progress is being made towards more responsible, equitable and impactful approaches to research, with examples including the Responsible Research in Business and Management network5 and EFMD’s Business School Impact System (BSIS)6, focused on accelerating research to the benefit of society, measuring and rewarding the societal impact of the research itself.
Given the ambition of all of this work and the desire to broaden research evaluation beyond journal articles, conference proceedings and book outputs, one might ask whether publishers have a place in co-creating the future of research evaluation. However, Emerald’s vision for equity focuses on helping to broaden participation in research and publishing as a facilitator towards real impact in society. As an independent publisher, our core competencies include helping business school authors optimise the reach of their trusted research, so clearly, it is a fact that we publish research outputs. However, Emerald’s contribution to The Hidden REF is guided by the desire to publish impactful content that can contribute to positive change in society rather than the pursuit of ‘REF-able outputs’ for the UK or any other evaluation system across the world. So, our objective is not to publish more journal articles as an end in itself rather we are excited to work alongside our social science author community to create new forms of digital content that engage new audiences in different ways, appended to or associated with the underlying research so that it is useable without detriment to the rigour of the research itself. The research ecosystem is complex, and research culture is notoriously slow to change, but the development of The Hidden REF, its Festival and Competitions indicates a strong appetite for change.
Through our collective learning, we are also developing tools and services to support the research community in its approach to responsible research, publication and evaluation. The foundation for all of this is close collaboration, embodied by our call for collective action, ‘Are you in?’7 After all, business school research is ultimately about solving managerial problems and realising real-world benefits by helping students to become insightful, effective decision-makers. As the approach to research evaluation becomes more equitable, it’s going to take continued, concerted collaboration to bring about a faster shift in research culture, and we invite readers to join this sector-wide effort.
Footnotes
1 The first Hidden REF Competition 2021 Hidden REF Competition – The hidden REF (hidden-ref.org)
2 UK Research Excellence Framework (REF 2029) REF 2029
3 Derrick, G.E., S. Hettrick, J. Baker, E. Karoune, S. Kerridge, G. Fletcher, N. Chue Hong, L Ballantyne, J. Fransmann, T. Roche (April 2024) The Hidden REF White Paper. Emerald Publishing Ltd, DOI:10.1108/EGP-02-2024- 0001 The Hidden REF White Paper – The hidden REF (hidden-ref.org)
4 The second Hidden REF Competition 2024 Hidden REF Competition 2024 – The hidden REF (hidden-ref.org)
5 Responsible Research in Business and Management (RRBM) RRBM network – Responsible Research in Business & Management
6 Business School Impact System (BSIS) Business School Impact System (BSIS) – EFMD Global
7 Emerald Publishing ‘Are You In?’ Are you in? | Emerald Publishing (emeraldgrouppublishing.com)
Tony Roche ([email protected]) is Chief Officer for Publishing and Strategic Relationships at Emerald Publishing. He serves as a Committee Member for The Hidden REF, a Member of Business In The Community UK’s Yorkshire and Humber Regional Leadership Board, and is Emerald’s Champion for the United Nation’s Sustainable Development Goals. He also acts as Emerald’s Head of House for the STM industry body, advancing trusted research.
