KAPTUR conference, London, March 2013
Posted: March 7, 2013 Filed under: events | Tags: CKAN, EPrints, jiscmrd, RDM policies, technical analysis, ukdcc Leave a comment
Working in Stages with DataStage and Figshare
Posted: October 20, 2012 Filed under: technical | Tags: DataFlow, DataStage, EPrints, Figshare, technical analysis 1 CommentWith thanks to Carlos Silva, Technical Manager, for the following blog post:
The KAPTUR Technical Analysis report (PDF) recommended the piloting and further investigation of two different systems: DataStage to EPrints; and Figshare to EPrints.
Figshare to Eprints
Some of the advantages of integrating Figshare with EPrints are:
- The upload tool to Figshare allows multiple uploads using WebDAV and javascript.
- The Figshare team is currently working on a desktop uploading tool to allow users a streamlined process of submission.
- Feedback from the Steering Group was that the user interface of Figshare was attractive and clear; it is already being used by researchers to store and manage research data and therefore the integration with EPrints would enable many institutions (as EPrints is the major repository platform in the UK) to encourage researchers to better manage their research data and then upload selectively to an institutional repository for publication.
Following telephone and Skype chats with the Figshare team a requirements document was created and shared with project partners and Simon Hodson. The idea was to create an API which would be free for use by any institution who wanted to link Figshare with an EPrints repository using the SWORD 2 protocol. Additional features included the development of the desktop uploader; a custom user interface design; back-end application development; and custom user accounts for the KAPTUR project partners to test the system.
Currently, negotiations are still in progress and further thought has been given to the infrastructure and pricing models that will eventually have an impact when adopting a commercial approach with technologies such as Figshare and that if not considered could lead to an unsustainable solution for the sector.
DataStage to EPrints
The second pilot recommended by the report was to link DataStage (from the JISC funded DataFlow project* with EPrints. The technical implementation of this pilot started in June 2012 when the Technical Manager set-up DataStage and DataBank on a local machine; demonstrated this to the Project Officers (in June) and the Steering Group (in July) and started collecting feedback on this. After testing the DataFlow software internally, the team started to explore the best way of linking DataStage with EPrints directly.
The advantages of integrating DataStage with EPrints are:
- DataStage offers the potential of being institutionally based, and therefore tighter control.
- It provides a structured metadata collection interface.
- It also provides flexibility when uploading, for example with the integration of a shared drive which uses a popular storage approach similar to Dropbox but with the advantages that the data is held on the institution’s servers.
Proposed integration of DataStage with EPrints, July 2012 (SlideShare)
The Technical Manager through VADS’ host institution – the University for the Creative Arts – set-up a test environment for the KAPTUR project (http://kaptur.ucreative.ac.uk). Test accounts have been given to project partners and an online feedback form set-up to capture this information.
To test the DataStage connection with EPrints, a test repository with the latest EPrints version (3.3.10) was needed in order to use the SWORD 2 protocol; this was created (http://kaptur_repo.ucreative.ac.uk).
Both systems have been tested separately, and both systems have performed well.
The DataStage software should allow users to submit entire folders as ‘packages’ to a repository using the SWORD2 protocol, however currently there is an issue** with the default version of DataStage and no transfers can be done on any other repository other than into Databank (the DataFlow project’s repository).
As well as contacting DataFlow and EPrints, the Technical Manager has been in contact with various colleagues across the sector, from the Centre for Digital Music at Queen Mary, University of London (see blog post about connecting DataStage with DSpace) to other colleagues who have also looked into connecting DataStage with EPrints such as the UK Data Archive, University of Essex and the RoaDMaP project, University of Leeds.
At this point there are the following conclusions:
- EPrints 3.3 is required in order to have SWORD 2 fully enabled [completed].
- EPrints have tested the SWORD 2 protocol successfully with other EPrints repositories, however connectivity with other types of repositories hasn’t been tested by EPrints yet.
- The DataFlow project manager replied saying that there were issues with the SWORD submission on the DataStage side, however they were expecting to come up with a workaround for their V 1.0 release [It is noted that Richard Jones will be presenting about DataFlow at the JISCMRD Nottingham programme event so this is hopeful!!].
- The lead DataStage developer mentioned that SWORD2 was envisioned to fully work with DataStage and EPrints when it becomes available and that previous versions of DataStage managed to work okay with EPrints, however due to new developments and enhancements at either end some changes in the DataStage side need to happen before it fully complies and can connect with EPrints.
*DataFlow was funded by JISC, under the University Modernisation Fund, from June 2011 – May 2012 to further develop a prototype out of the JISC-funded ADMIRAL project (2009-11).
KAPTUR ten months into the project – (10/18)
Posted: August 20, 2012 Filed under: monthly reports, project posts | Tags: idcc13, jiscmrd, technical analysis, ukdcc Leave a commentThis is our update for the end of the tenth month:
WP1: Project Management
- The second steering group meeting was held in late July in London.
WP3: Technical Infrastructure
- The Technical Manager provided a demonstration of DataStage and Figshare at our steering group meeting in July; feedback from partners was very useful in informing the development of the KAPTUR pilot infrastructure.
WP4: Modelling
- The UAL Project Officer has been continuing with the DCC University Engagement programme and interviewing arts researchers face-to-face using an online survey developed with Marieke Guy and Dr Paul Ryan. This will inform their RDM policy development.
- There are now draft RDM policies for Goldsmiths, University of London; The Glasgow School of Art; and the University for the Creative Arts.
- Development of The Glasgow School of Art and the University for the Creative Arts draft RDM policies will continue in September when the University term resumes.
- The Goldsmiths Project Officer reported that on 25th July an informal working group convened including representatives from a variety of departments (academic, support, management) and Martin Donnelly from the DCC. After presentations by Martin, Jacqueline Cooke, and the Project Officer, a draft policy prepared by the Project Officer was discussed as well as requirements for putting it into action. Martin provided extremely useful and relevant information for everyone involved. The proposal for an RDM policy was well received. The group (though perhaps not in the exact original composition) will reconvene in the autumn with a view to finalising a commitment relating to RDM to be tabled at meetings at the end of the year.
WP5: Training and Support
- This workpackage is due to begin in September, however it was necessarily an item on the agenda of the steering group meeting in July.
- During August the Project Manager has been researching existing training materials, for example from previous JISCMRD funded projects, in preparation for discussion at the next project team meeting in September, including using Pinterest here: http://pinterest.com/mtgwork/rdm-training/
- In particular, in response to feedback from the steering group, a selection of 45 RDM videos have been bookmarked here: http://pinterest.com/mtgwork/rdm-videos/
- The event on 14th September will also be an opportunity to get feedback on training materials and inform the development of the KAPTUR toolkits.
WP6: Evaluation and Sustainability
- This workpackage is due to begin in September, however it was necessarily an item on the agenda of the steering group meeting in July.
- In response to discussion at the steering group, the Technical Manager has been working with the Project Director and Project Manager on a costings template and method for the project partners. This will enable costs to be estimated for visual arts technical infrastructure at each institution over the next 5-10 years, and lead into the Business and Sustainability plans.
WP7: Dissemination
- The Project Manager has continued to work with Angus Whyte of the DCC and Laura Molloy, JISC Evidence Gatherer, on the ‘Selecting and Appraising Research Data’ event to take place in London on 14th September. This has now been renamed ‘Managing the Material: Tackling Visual Arts as Research Data’ and is almost fully booked. The last few places, and more details, are available here: http://kapturmrd01.eventbrite.co.uk/
- The UAL Project Officer will be presenting about KAPTUR at the UAL Information Services Staff conference on Friday 21st September.
- The Project Director has, with the Technical Manager and Project Manager, written and submitted a paper to the 8th International Digital Curation Conference which builds upon the research undertaken as part of the Technical Analysis report
Technical development update
Posted: June 12, 2012 Filed under: technical | Tags: DataFlow, DataStage, EPrints, technical analysis 1 CommentSince the beginning of the KAPTUR project, the Technical Manager has maintained contact with the UCA IT department to ensure they are aware of the project and its requirements. Work requests to IT have established precise deadlines, however for the purposes of this blog post the following tasks are represented in a month-by-month format for easy viewing:
| Who | What | When (2012) | ||||
| June | July | August | September | October | ||
| KAPTUR Technical Manager | Download and install DataFlow on a local environment (not server). | X | ||||
| KAPTUR Technical Manager | Once DataFlow is stable, setup EPrints in the same local environment. | X | ||||
| KAPTUR Technical Manager | Development work to link DataFlow with EPrints. | X | ||||
| KAPTUR project team and partner institutions | First round of feedback and tests. | X | ||||
| UCA IT Department | Create Virtual Machine (VM) on servers hosted by UCA. | X | ||||
| UCA IT Department | Assign the URL kaptur.ucreative.ac.uk to the VM. | X | ||||
| UCA IT Department | Create back-up mechanisms in line with usual procedures for the VM. | X | ||||
| KAPTUR Technical Manager | Transfer local development environment to the new VM. | X | ||||
| KAPTUR Technical Manager | Test and debug. | X | ||||
| KAPTUR project team and partner institutions | Further feedback and tests of the system via the VM. | X | ||||
| KAPTUR Technical Manager | Further tests and debugging leading to initial pilot system. | X | ||||
The above table is based upon the original KAPTUR workpackages, additional investigations undertaken as part of the Technical Analysis report, and ongoing consultation with the project team.
Building a pilot demonstrator service for the visual arts
Posted: May 28, 2012 Filed under: resources, technical | Tags: DataFlow, DataStage, DSpace, EPrints, Fedora, Figshare, jiscmrd, technical analysis Leave a commentThe following blog post is adapted from the Conclusion and Recommendations section of the Technical Analysis report (PDF):
The KAPTUR Technical Manager investigated 17 different types of software which were compared to the requirements of the four partner institutions (details and appendices in the report). The next stage of the research reduced the choice of software to five options: DataFlow, DSpace, EPrints, Fedora, Figshare. These were all found to be suitable for managing research data in the visual arts; through a further selection process EPrints, Figshare, and DataFlow were identified as the strongest contenders.
[…] it is recommended that two pilots occur side by side: an integration of EPrints with Figshare and a separate piece of work linking DataFlow’s DataStage with EPrints. By integrating EPrints with Figshare, the project can take advantage of a system which has been built with, and for, researchers to handle research data specifically, and has a user-friendly visual interface (which is constantly evolving and enhanced by Figshare directly). […]By integrating DataStage with EPrints the research data storage and software will be hosted within each institution, providing them with better control over the type of data that can be stored, published and managed. The integration will also enable content uploaded in DataStage to be securely backed up by the institution and accessible from anywhere in the world. A ‘Dropbox’-like tool is featured in the latest beta version, providing a user-friendly interface which will benefit visual arts researchers. EPrints will effectively provide the role of DataFlow’s DataBank.
KAPTUR 