Institutional Research Data Management at Newcastle University
March 25, 2013 Leave a comment
The iridium project (@iridium_mrd) is attending the Jisc MRD02 Achievements, Challenges and Recommendations Programme Workshop March 25-26 2013.
Ben Allen will be presenting on observations of the technical landscape.
We will also present a poster on key project outputs.
iridium_Jisc_final_meeting_25_3_2013_sml
November 1, 2012 Leave a comment
The iridium project presented at the JISC MRD02 Progress Meeting in Nottingham. The two day schedule from the event is here, together with the Programme introductory/close slides.
Workshop topics were:
Individual projects were encouraged to contextualise presentations around the following themes:
[1] “what has worked/is working”
[2] “what lessons you have learned and how generalisable these may be”
[3] “what challenges remain”
[4] “how such challenges may be approached and what your institution/project intends to do”
[5] “what DCC / MRD activity you think may help make the challenge more tractable”
iridium ‘support’ presentation within ‘Training & Guidance’ session:
iridium presentation
iridium_JISC_Progress_25_10_2012_v4_web_sml_LW [.pdf]
We also presented two posters, one on the research data catalogue proof-of-concept and the second on our thematic analysis requirements gathering.
Other project presentations from the Programme are available here.
October 24, 2012 Leave a comment
For more details, see full survey report: http://research.ncl.ac.uk/media/sites/researchwebsites/iridium/iridium_online_survey_report_17_8_2012_v2.1_SK.pdf
October 24, 2012 Leave a comment
Janet Wheeler and I will be attending the JISC Progress Meeting/DCC event in Nottingham and are presenting a poster on developing an RDM tool to support implementation of policy principles.
JISC MRD Progress Meeting poster
October 24, 2012 Leave a comment
iridium project team members from the Digital Institute presented a poster on the JISC-funded thematic analysis work carried out with local research and related staff to gather and understand RDM requirements. The work was present at the recent Digital Research 2012 conference in Oxford.
iridium DI Digital Research 2012 poster
October 4, 2012 3 Comments
What follows is a summary of my steps along the path of investigating what the sword technology is, through to being able to actually start to code something useful; I should probably point out that the beginning of this post can be consumed by less technical persons as a quick overview, but the later section assumes that you…
(Since my investigations centered around sword in conjunction with Sakai and e-science central my language of choice therefore is Java).
I should also declare that I still don’t fully understand all of the implementation but this should help you along your way if you’re just starting out!
My first port of call was the SWORD website itself, which will point you to some useful videos and slides to give you insight into what the technology is and what it can be used for. In short, this is what the “Sword Course” will teach you…
What it is:
The “Simple Webservice Offering Repository Deposit” technology (or SWORD for short) intentionally only deals with how to get data into a repository, nothing else, and is complementary to something like dublin core used to describe stuff in a repository; it also does not deal with packaging, metadata, authentication or authorisation
Existing implementations can be found in:
DSpace
Eprints
Fedora
Intralibrary
Zentity
Use cases sword is trying to enable:
When you use a sword client, this is basically what happens…
What sword adds to ATOM Pub:
There are generally three sorts of client:
Interesting possibilities for deposit scenarios:
My next stop on the journey through sword looked at actual code, how it’s laid out and what you need to do in order to start doing something useful.
As mentioned previously, I have been basing my investigations around the Java client and server libraries but I’d strongly recommend you also get a good grounding in the workings of ATOM Pub (HTML Version) and the Sword Profile specifications themselves. If you’ve ever read specification documents before you’ll know they can make quite dry reading, however, since ATOM Pub and sword are relatively straight forward technologies and the specs only reach into 30-50 odd pages it’s really worth a browse through.
Firstly, it’s probably best to understand a few basic concepts you will need to deal with, the main outputted concepts/objects are:
Next, I found you learn the most by studying the server libraries, what you get is a bundle of java and some set up files for your container. We’ll firstly look at the setup in web.xml
The main servlets (i.e ultimately your rest endpoints) that are defined are…
servicedocument
collection
mediaresource
container
statement
The code makes heavy use of interfaces to allow the implementer more freedom to create functionality using the server library in the way they want to, in order to tell the server libraries what code we want to instantiate and what auth. mechanism we will be using, you must set some context parameters to define those settings and implementations used at runtime:
auth
You can set an Authorization header and base64 encode user:password (e.g. try going to a basic auth encoding website and encode “user:password” in plain text box) and send as a header… “Authorization” “Basic dXNlcjpwYXNzd29yZA==”, or, if you prefer set the param-value to “None” for no authorisation. I found (at the time of writing) the default code actually has a bug which means turning the auth off doesn’t work correctly. I found the best way of correcting this was in my own war project (which includes the server libraries as a dependancy) I created a org.swordapp.sever package (where I was implementing the objects needed for the interfaces) and dropped in a copy of the SwordAPIEndpoint.java to override the implementation in the library, I then changed the getAuthCredentials to…
protected AuthCredentials getAuthCredentials(HttpServletRequest request, boolean allowUnauthenticated) throws SwordAuthException
{
AuthCredentials auth = null;
String authType = this.config.getAuthType();
String obo = "";
this.log.info("Auth type = "+authType);
//If we are insisting on "a" form of authentication that is not of type "none"
if(!allowUnauthenticated && !authType.equalsIgnoreCase("none"))
{
// Has the user passed authentication details
String authHeader = request.getHeader("Authorization");
// Is there an On-Behalf-Of header?
obo = request.getHeader("On-Behalf-Of");
// Which authentication scheme do we recognise (should only be Basic)
boolean isBasic = authType.equalsIgnoreCase("basic");
if(isBasic && (authHeader == null || authHeader.equals("")))
{
throw new SwordAuthException(true);
}
// decode the auth header and populate the authcredentials object for return
String[] userPass = this.decodeAuthHeader(authHeader);
auth = new AuthCredentials(userPass[0], userPass[1], obo);
}
else
{
log.debug("No Authentication Credentials supplied/required");
auth = new AuthCredentials(null, null, obo);
}
return auth;
}
The following context parameters set the implementations of interfaces used to implement functionality in the endpoints. However, you are not given default implementations for each of these, so in your war you need to create a new class that implements the respective interface and fill in your functionality….
collection-list-impl
service-document-impl
collection-list-impl
collection-deposit-impl
media-resource-impl
container-impl
statement-impl
config-impl
Let’s now have a look at the servlets themselves, each servlet contains interfaces which are implemented by loading the classes specified in the web.xml (see above).
All servlets used in the server library extend the “SwordServlet” which (obviously) extends the HttpServlet. Since the SwordServlet contains the server configuration object, all servlets (through inheritance) also hold an implementation of the server configuration (i.e. SwordConfiguration) and a method to allow servlets to load classes from the configuration….
SwordServlet encapsulates:
CollectionServletDefault extends SwordServlet and encapsulates:
ServiceDocumentServletDefault extends SwordServlet and encapsulates:
MediaResourceServletDefault extends SwordServlet and encapsulates:
ContainerServletDefault extends SwordServlet and encapsulates:
StatementServletDefault extends SwordServlet and encapsulates:
Those with a keen eye will have noticed that each servlet is also holding an “API” object, these objects fill out the standard Get/Post/Put/Delete HttpServlet methods that the servlets override by taking the configuration object and any interfaces that have been implemented and combine them to do something useful. Similarly to the servlets, they all extend a Sword API super class called “SwordAPIEndpoint”, which holds a SwordConfiguration implementation. The hierarchy (and interfaces they encapsulate) looks like this…
SwordAPIEndpoint
CollectionAPI extends SwordAPIEndpoint
ServiceDocumentAPI extends SwordAPIEndpoint
MediaResourceAPI extends SwordAPIEndpoint
ContainerAPI extends SwordAPIEndpoint
StatementAPI extends SwordAPIEndpoint
I keep mentioning the objects that implement the interfaces, I thought it might be useful to go through in “slightly” more detail what the content of those objects are intended for. Apologies once again, this is not exhaustive as I have not worked my way through what all the methods are intended for:
SwordConfigurationDefault implements org.swordapp.server.SwordConfiguration
CollectionListManagerImpl implements org.swordapp.server.CollectionListManager
CollectionDepositManagerImpl implements org.swordapp.server.CollectionDepositManager
ServiceDocumentManagerImpl implements org.swordpapp.server.ServerDocumentManager
MediaResourceManagerImpl implements org.swordapp.server.MediaResourceManager
ContainerManagerImpl implements org.swordapp.server.ContainerManager
StatementManagerImpl implements org.swordapp.server.StatementManager
Once you have all that setup (and I’d recommend just creating skeleton override methods for the objects implementing interfaces for the time being whilst you figure the code out), you can then start coding the abdera / sword code and try make the client do something. The client itself comes with a handy cli driven (SwordCLI) interface that you can point at your newly created server instance and test the various example methods. I would recommend though, that you comment out the entire list of method references in the main method and go through the list iteratively to slowly make each part of your server work…
As a brief example, if we were to try and get a basic service document to work, try adding this code to your ServiceDocumentManagerImpl.java….
public ServiceDocument getServiceDocument(String sdUri, AuthCredentials auth, SwordConfiguration config) throws SwordError, SwordServerException, SwordAuthException
{
//Our test service document
ServiceDocument sd = new ServiceDocument();
//sd.setVersion("2.0");
sd.setMaxUploadSize(1000000000);
//Our test workspace
SwordWorkspace sw = new SwordWorkspace();
sw.setTitle("TestWorkspace");
//Our test collection
SwordCollection sc = new SwordCollection();
sc.setHref("http://<yourServer>/<yourWebapp>/collection/TestCollection");
sc.setTitle("TestCollection");
sc.addAccepts("*/*");
sc.setCollectionPolicy("TestCollectionPolicy");
sc.setAbstract("TestCollectionAbstract");
sc.setMediation(false);
sc.setTreatment("A human-readable statement describing treatment the deposited resource has received or a URI that dereferences to such a description.");
sc.addAcceptPackaging("http://purl.org/net/sword/package/SimpleZip");
sc.addAcceptPackaging("http://purl.org/net/sword/package/METSDSpaceSIP");
sc.addAcceptPackaging("http://www.ncl.ac.uk/terms/package/html");
sc.setLocation("http://<yourServer>/<yourWebapp>/collection/TestCollection");
sc.setMediation(true);
List iris = new ArrayList();
iris.add(new IRI("http://<yourServer>/<yourWebapp>/collection/TestCollection/TestSubService"));
sc.setSubServices(iris);
//Add collection to workspace
sw.addCollection(sc);
//Add workspace to service document
sd.addWorkspace(sw);
return sd;
}
Browsing to http://<yourServer>/<yourWebapp>/collection/ should return something, and removing your comment in your client for the line…
cli.trySwordServiceDocument()
…should now yield some results (If you just get errors try temporarily turning off the auth. requirement on the server for the purposes of testing).
And that’s the basic principle, you then take the specs and implement the returning and unpackaging of ATOM using the abdera/SWORD objects and link what’s passed/returned to the content found in the system you are trying to integrate.
Sword Specs
Brief history of Sword
More useful Sword docs
Another set of slides explaining sword
Andrew Martin
Research and Collaborative Services
Newcastle University
August 16, 2012 Leave a comment
Upcoming national RDM events:
Call for Papers for International Digital Curation Conference
Deadline: extended to 30th (20th August 2012)
The call for papers is very broad and inclusive, so take the opportunity to share your good practice: www.dcc.ac.uk/events/idcc13/call-papers
DataCite Technical Workshop
11:00-15:00 10th September 2012, The British Library Conference Centre, St. Pancras, London, NW1 2DB
This practical workshop is aimed at those who are considering incorporating DataCite services into their repository and would like to learn more about how to work with the technology.
The event will be limited to 15 people. We will consider running additional sessions if it is oversubscribed.
Please bring your own laptop.
If you would like to register for the event, please reply to this address with the following details:
Name:
Affiliation:
Project (e.g. repository name or JISC-MRD involvement):
Role:
Caroline Wilkinson
Data Management Project Officer
Science, Technology and Medicine
The British Library, 96 Euston Road, London NW1 2DB
Tel: 020 7412 7250
Email: caroline.wilkinson@bl.uk
Web: www.bl.uk/datasets
Mendeley Group: bit.ly/BLdatacitation
‘Managing the Material: Tackling Visual Arts as Research Data’ Workshop
09:45-16:00 14th September 2012, HEFCE, Centre Point, London WC1A 1DD
Further details and registration is available from: http://kapturmrd01.eventbrite.co.uk/
SPRUCE Mashup
18th-20th September 2012, The Hubworking Centre, 5 Wormwood Street, EC2M 1RQ, London, United Kingdom
If you’d like to attend the SPRUCE Mashup London, please sign up here: http://sprucelondon.eventbrite.co.uk/
SPRUCE is a JISC funded Project. For more information see: http://wiki.opf-labs.org/display/SPR
Call for Papers/Tools for 4th Annual European DDI User Conference (EDDI12) DDI – The Basis of Managing the Data Life Cycle
Deadline: 3rd October 2012
Seeking both papers and tools on all things DDI. If you are interested in presenting a paper, please use the online submission system<http://www.eddi-conferences.eu/ocs/index.php/eddi/eddi12/schedConf/cfp> of the conference. The deadline for submissions is October 3, 2012. Please use the same link to submit an abstract on your tool until October 3, 2012. The deadline for submissions of the final program code is November 17, 2012.
2nd Workshop on Semantic Statistics for Social, Behavioural, and Economic Sciences: Leveraging the DDI Model for the Linked Data Web
15th-19th October 2012, Leibniz Center for Informatics, Schloss Dagstuhl, Wadern, Germany
Further information on the workshop, including venue details<http://www.gesis.org/fileadmin/upload/dienstleistung/veranstaltungen_fortbildungen/workshops/Dagstuhl.pdf> and a registration form, is available on the website of the workshop: http://www.gesis.org/en/events/gesis-workshops/ddi/.
June 27, 2012 1 Comment
Niall O’Loughlin will be presenting a poster from the iridium project at the ECRM12 (http://academic-conferences.org/ecrm/ecrm2012/ecrm12-home.htm) conference.
iridium RDM poster presentation at ECRM12 Bolton
Available to download as PDF file iridium_ECRM12_poster_low_res.
June 12, 2012 1 Comment
iridium will be presenting at the Association of Research Managers and Administrators annual conference , ARMA 2012, on 12-13 June (http://www.arma.ac.uk/news/conferences/index.xhtml).
Copy of poster is below:
iridium research data management poster ARMA
Poster PDF available to download here: iridium_ARMA_2012_low_res_v1_LW.
March 8, 2012 Leave a comment
From the DCC website:
DCC Roadshow Northeast England
Newcastle upon Tyne, 23-24 April 2012
“The 10th DCC regional roadshow will take place on 23rd and 24th April at the Darwin Suite, Centre for Life Conferencing Centre, Life Centre, Newcastle upon Tyne, NE1 4EP. The roadshow is being co-organised with Professor Julie Mcleod, Northumbria University School of Computing, Engineering, and Information Sciences. Registration is FREE and delegates can choose to attend one or both days, according to their roles and responsibilities.”
http://www.dcc.ac.uk/events/data-management-roadshows/dcc-roadshow-northeast
iridium 