Directory Ingest Service 1.2

Current released version

DirIngest 1.2 has now been released.

1. Overview

This service is part of the Fedora Service Framework. It constructs Fedora objects from uploaded Submission Information Packages ("SIPs") and ingests those objects into a Fedora repository. It is exposed as a REST-style web service that accepts input via HTTP POST and (if successful) returns an XML document with the list of PIDs of the ingested objects.

The following diagram illustrates:

2. Installation

2.1 Prerequisites

A running Fedora repository, version 3.0.
A servlet container that supports the Servlet API v2.3+

2.2 Setup

Note: These instructions are written assuming you are using Tomcat. Web application deployment may differ for your servlet container.

First, copy the diringest.war file to the "webapps" directory and re-start Tomcat. The war file should be deployed into "webapps/diringest" by the Application Server.

Edit the "WEB-INF/classes/diringest.properties" file, changing the values to what is appropriate for your installation.

#
# Required settings for remote Fedora repository.
# These are used for the pid generation and ingest functions.
#
fedora.baseURL = http://localhost:8080/fedora/
fedora.user    = fedoraAdmin
fedora.pass    = fedoraAdmin
#
# Optional preferred namespace part for PIDs.
# Comment this out to use the default from the target Fedora repository
#
pid.namespace = demo

Optional: If desired, modify the default conversion rules file in this directory (crules.xml). See the section below on specifying conversion rules.

Optional: If desired, modify the logging configuration file in this directory (log4j.xml).

Finally, ensure that your Fedora server is running, and re-start Tomcat.

3. Using the Service

Once installed, the service will be exposed from your servlet container at "diringest/ingestSIP". For example:

http://localhost/diringest/ingestSIP

Navigating the URL above using your browser will present a simple HTML form where you can upload a SIP file, an optional rules file (if you want to use non-default conversion rules). Select "Submit". A successful submission will result in an XML file that contains a list of the PIDs of the objects that were ingested.

The service accepts, via HTTP POST, the following parameters:

sip - The SIP file. See Preparing a SIP for format details.
rules (optional) - A custom conversion rules file. See Specifying Conversion Rules for format details.

A successful response will return a 200 (OK) HTTP status code and a simple XML file containing PIDs, such as the one below:

<?xml version="1.0" encoding="UTF-8"?>
<pidList>
  <pid>demo:42</pid>
  <pid>demo:43</pid>
  <pid>demo:44</pid>
  <pid>demo:45</pid>
</pidList>

4. Using the SIP2FOX Utility

The "SIP2FOX" utility is useful for command-line testing of SIPs and conversion rules. It is included in the distribution's "sip2fox" directory. To invoke it, change to that directory and type:

sip2fox sipFile.zip outputDir [fedoraHost fedoraPort fedoraUser fedoraPass [pidNamespace]]

If you only specify a SIP file and an output directory, the program will use a test PID generator for creating the objects. If you specify Fedora Repository login information, that repository's PID generator will be used instead. To use a PID namespace other than the default for a repository, specify it as the last parameter.

The SIP2FOX utility uses the rules that reside in the config/crules.xml when it runs.

5. Preparing a SIP

A SIP file is a ZIP file containing metadata and content for a set of to-be-created Fedora objects.

It consists of a single "Manifest.xml" file in METS format (actually a subset of METS), and 0 or more datastream files that are referenced from the METS file.

Sample SIP files and a readme.txt file with descriptions of each are included in the sip2fox/samples directory in the diringest distribution.

6. Specifying Conversion Rules

A conversion rules file is used by the service (or sip2fox) in deriving RELS-EXT relationships from the hierarchical structure (the structMap) in the SIP.

The XML schema for conversion rules is included in the diringest distribution's sip2fox/config/ directory. There you can also find the default crules.xml file, which illustrates the use of several templates.