#jiscmrd – Kaptur’s post on benefits and metrics #KRDS

Simon Hodson, JISCMRD Programme Manager, has asked all 18 month JISCMRD projects to write a blog post about the key expected benefits that each project will achieve, and what metrics we will use to evidence these at the end of the project.

Links

• Report on the Benefits from the Infrastructure Projects in the JISC Managing Research Data Programme
• The KRDS Benefits Analysis Toolkit and specifically: Introduction to the KRDS Benefits Analysis Toolkit; Guide to the KRDS Benefits Framework Tool; Guide to the Value Chain and Benefits Impact Tool

Following a presentation by Neil Beagrie, Director of consultancy at Charles Beagrie, the JISCMRD projects were provided with a ‘Summary of Benefits Identified by the RDMI Projects’ and a ‘Summary of Metrics Identified by the RDMI Projects’. We were invited to select three benefits and then match these up with the appropriate metrics, making sure to include both quantitative and qualitative metrics for each benefit. I would like to emphasise that the following has not been discussed within the project team yet and is subject to confirmation.

Benefits

1. Sustainability of research data infrastructure.
2. Change to user practices.
3. Mitigating organisational risks.

Metrics

1. By each institution creating and approving its own Business Costs and Sustainability plans; the ultimate proof is in the longevity of the research data infrastructure. Qualitative data will be gathered through the Steering Group meetings which will include high-level senior staff across the four institutions. Quantitative data may include percentage or estimated cost savings/efficiencies for central services and/or departments.
2. By taking a snapshot of existing practice at the four institutions through the Environmental Assessment report and then through maintaining user engagement throughout the project and taking snapshots at key stages to monitor progress. Qualitative data will be gathered through the interviews undertaken as part of the Environmental Assessment report, and through ongoing engagement e.g. through working groups and/or focus groups. Quantitative data will be gathered in the following ways: online questionnaires and/or feedback forms to record the impact on working practice of the project, these would be undertaken at key points e.g. we are planning an online survey in January, and would also gather feedback after training events; if online training materials are created then usage statistics will be gathered.
3. By taking a snapshot of existing practice at the four institutions through the Environmental Assessment report and then through maintaining user engagement throughout the project and taking snapshots at key stages to monitor progress. Qualitative data will include a range of stakeholder examples of improved risk management e.g. organisational practice before Kaptur and how this has changed during/afterwards. Quantitative data may include a percentage improvement in routine back-up of data, and/or a percentage improvement in research data management awareness and policies/systems.

Neil emphasised that when using the KRDS tools it is a good idea to do initial work by one individual and then work this up in a project team context. Therefore the points raised in this blog post will be discussed at our next meeting in early January and possibly again at the Steering Group meeting. Neil also mentioned that it was important to adapt the tools to your project needs. From reading the documentation I am also aware of the need to start with the benefits framework tool prior to moving on to the value chain and benefits impact tool. The work done by other projects giving example worksheets is really useful, in particular the UK Data Archive and the Archaeology Data Service. Reference: Report and Presentations from the JISC Digital Curation/Preservation Benefits Tools Project Dissemination Workshop