The first in-person meeting of national Reproducibility Network (RN) representatives took place in the Berlin Institute of Health Quest Centre in May 2024, and we thank Ulf Tölch and his colleagues for their warm welcome and excellent hospitality.
Representatives from 10 RNs (Austria, Brazil, Belgium, Croatia, Germany, Italy, Norway, Spain, Switzerland, UK) met in person, with several more country and regional representatives (Africa, Belgium, Brazil, France, Netherlands, Slovakia) joining part of the meeting online. The sessions were chaired by UKRN’s Marcus Munafò. We discussed how to best work together and deepen collaborations between this global community of national RNs, and agreed our next steps.
The first session revolved around ‘who we are’. National RNs promote a collaborative ethos between research institutions, funders and publishers with the legitimacy of being plugged into networks of grassroots researchers. We want to be able to do this globally, especially as publishers and some funders are global in reach. In terms of what makes RNs distinct, several representatives stressed the grassroots nature of their organization, connecting the ‘bottom up’ with the ‘top down’. There was also a distinction made between people-focused community activities and the more task-driven activities of RNs.
Defining reproducibility
One goal is to agree a definition of reproducibility, with an example being ‘the ability to reproduce the research process, making it as transparent as possible enables others to view the process and challenge the assumptions made’. The definition could broadened with input from different disciplines. And although challenging, it is a responsibility to listen to research communities that are not engaging with open research. A recent survey by members of the Croatian RN found that transparency emerges as the core term for qualitative researchers. The concept ‘positionality’ is routinely used by researchers in the humanities and social sciences, and this needs to be brought into discussions around defining reproducibility. There was a proposal for all RNs to add a definition of reproducibility to their terms of reference. There was no appetite from RN representatives to remove the term ‘reproducibility’ or re-name themselves; however, there needs to be strategy for how best to broaden the definition of reproducibility.
What are we doing
Current RN activities were discussed, including a Belgian RN survey of lecturers on how to incorporate reproducibility content into the curriculum. UKRN has approached the seven UK Government Research Councils offering to work with them to define reproducibility. UKRN has also been awarded UK government funding to run a Community Project. This fills gaps within UKRN communities (Local Networks, Institutions and Stakeholders), providing networking opportunities and resources for community building. The project also supports the international network of RNs and funded this in-person meeting. The German RN are considering using reproducibility ‘readiness levels’ to identify which institutions are ready for which interventions. Strategies and policies will be different for each level, and differential strategies could be developed. One of the lessons learnt within the German RN is that there needs to be a value proposition – for example, using case studies to demonstrate that reproducibility creates value in public-private partnerships. The Brazilian RN have been working alongside their national government to disseminate information on reproducibility, therefore making it easier to reach lots of institutions. They have also provided recommendations on how to evaluate or reward reproducibility to the national agency that analyses graduate programmes. There is broad scope for RNs to provide support and advice on reproducibility to their national funding agency. The Brazilian structure could provide a model here for other RNs.
Where are we going
In the ‘where are we going’ session representatives discussed what type of framework would best serve the global RN community, enabling it to perform its core business of engaging with researchers, funders, governments and institutions. It was agreed that although there are different sizes and degrees of maturity across RNs, there is no need for standardisation between RNs (for example, the number of representatives for each RN). Whilst representatives were broadly happy with online meetings there is much to gain from in-person events. We therefore proposed an annual in-person/hybrid RN meeting that could either coincide with existing conferences or be held in Europe where many RNs are based. It was agreed that RN meetings continue to be conducted in English and that it would up to local RNs to decide whether to translate UKRN resources into local languages, as the Croatian RN have done. There was discussion around how best to share information and updates between RN teams and how to make better use of the regular online meetings.
Developing a position paper
RNs have already been working on a strategy document for a loose federation and it was agreed that we will further develop this into a position paper setting out our history, our definition of reproducibility, and our RN aims.
Resourcing the global network of networks
Together the RNs want to create a global network of RNs and there was discussion around the resources required to achieve and maintain this. A global RN, with a centralised website, may be more attractive to potential funders such as philanthropic organisations. There was discussion around where a global RN would be located with part of the function of a global RN being to hold and distribute funds on behalf of RNs that are not legally registered. There was also discussion around bringing large reproducibility organisations together at a global level to ask how we can all work collaboratively.
Sustainability options
The final session included discussion around sustainability options and opportunities for RNs. UKRN are working to identify the exclusive and unique benefits of being part of a RN, these will be used to develop a sustainability model.
Training was a key theme in these discussions, with a goal of consistent training at the national level. This could also be applied at a global level, with national training from a global body providing consistency. There is scope to provide RN income through the delivery of training since many organisations already pay for training. To better understand what RNs could charge for, we need to identify the current range of activities across RNs.
Individual RNs are taking various approaches to sustainability. For example, the Swiss RN are trying out an institutional membership model with fees graded by the number of students. The German RN plan a similar model. The Italian RN have both personal and institutional membership fees. A Global RN may be sustained by subscriptions from RNs, and there is a moral need to cross subsidise RNs within a global RN.
Improving Reproducibility In SciencE (iRISE) workshop
The meeting was followed by an iRISE “improving Reproducibility In SciencE” training workshop led by Julia Prieß-Buchheit with discussion around two themes, what fascinates you about reproducibility and what is your current challenge.