Fedora Tutorial #2
Getting Started:
Creating Fedora Objects using the
Content Model Architecture
Fedora 3.0
July 23, 2008
Author: The Fedora Development Team
Copyright: ©2008 Fedora Commons, Inc.
Purpose: This tutorial introduces the basic development questions, design concepts and project goals of the Flexible Extensible Digital Object Repository Architecture (Fedora).
Audience: This tutorial is intended for repository administrators or content developers who will be using the Fedora software.
Table of Contents
Figures
Figure 1 – Fedora repository as Mediator for Services and Content
Figure 2 – Fedora Administrator Login Screen
Figure 3 – New Object Dialog
Figure 4 – Configuring an Object
Figure 5 – Datastream Display
Figure 6 – Adding a New Managed Content Datastream
Figure 7 – Complete Datastreams for Example 1
Figure 8 – Example 1 Digital Object and Datastreams
Figure 9 – Adding a Datastream with Type Redirect
Figure 10 – Example 2 Datastream Display
Figure 11 – Example Digital Object and Redirected Datastream
Figure 12 – Abstract View: Key Fedora Components for Producing Disseminations of Content
Figure 13 – Relationships Between Data objects and CModel/SDef/SDep Objects for CMA
Figure 14 – Dynamic Dissemination Access
Figure 15 – Example 3 Linking a Digital Object to a Content Model
Figure 16 – Example 3 Dissemination via CMA
Figure 17 – Dissemination with Redirect Datastream
This is an introduction for system developers and repository managers who are new to the Fedora Repository open-source content management software. This is a hands-on tutorial. It assumes that you have already installed the Fedora software and are at a computer with access to a Fedora repository through the Fedora Administrator
while reading this tutorial.
You don't have to have to be a programmer to understand and use this tutorial. However, you should be familiar with the operation and structure of web servers and web services.
This document is not intended for end users of content disseminated by a Fedora repository.
Fedora is content management software that runs as a web service within an Apache Tomcat web server. Fedora provides the tools and interfaces for creation, ingest, management, and dissemination of content stored within a repository. There are a number of features that distinguish Fedora:
A number of these features are illustrated in Figure 1.
Fedora may be the wrong choice for management of simple static web pages. There are a number of excellent tools for HTML editing and web site creation. Fedora is more appropriate for more advanced content management tasks. These include management of content and associated metadata, multiple versions of content, content available in multiple formats, and dynamically generated content from local and dynamic sources.
This document is intended to be hands-on, with you trying the examples on a running Fedora repository. You should therefore, have already downloaded and installed Fedora, and started a server. You should then access the Fedora repository by running the Fedora Administrator interface, fedora-admin
, which is located in the FEDORA_HOME/client directory
(you can start this program from the command line if you have configured your environment variables properly). Upon starting up the administrator interface you will be presented with the Login
screen shown in Figure 2. This document assumes that you have not changed any of the configuration defaults for your Fedora server so the password you enter should be fedoraAdmin
. If you have changed your configuration values or are running the Fedora Administrator from a machine different from the machine on which your Fedora server is running you will need to change the values in the Login
screen appropriately.
Figure 2 – Fedora Administrator Login Screen
You should read this document in order, since later examples assume knowledge of techniques and definitions introduced earlier.
The font conventions used are:
URLs, directory paths, file names, and similar items are shown like this.
All path names assume that you have set your FEDORA_HOME
environment variable and descend from the directory defined by that variable.
All URLs that access the Fedora repository assume that the host:port
of the repository is localhost:8080
.
This section describes how to create digital objects in Fedora that aggregate data from multiple sources. The examples demonstrate how to do this with both local data and data from networked sources. This section provides the foundation for the next section, which describes how to use Fedora to create dynamic content by exploiting web services. Make sure you understand the basic concepts here, before moving on to that next section
To understand content aggregation in Fedora, you need to be comfortable with two terms:
Decisions about what to include in a digital object and how to configure its Datastreams are basic modeling choices as you develop your repository. The examples in this tutorial demonstrate some common models that you may find useful as you develop your application.
It is often useful to provide access to a digital document in several formats. For example an ePrints server might provide HTML for those who wish to render the document in a browser, PDF for those who wish to view the document with author-determined formatting, and TeX for those who wish to access and use the document source. This example demonstrates how to construct a digital object where each Datastream corresponds to an available format. More advanced techniques, demonstrated later in this tutorial, make it possible to achieve the same results by generating formats dynamically from a single base format. But for now, we'll stick to simple static aggregation.
Start by selecting File/New/Data Object in the Fedora Admin GUI. Complete the New Object
dialog box as shown in Figure 3.
Figure 3 – New Object Dialog
Check the box for Use Custom PID
and enter demo:100. Note that when you do not assign your own PID, the Fedora repository will create one for you. Select the Create
button and you should see a window like that shown in Figure 4. Observe that the PID of the created object (in this case demo:100) is displayed in the title bar.
Figure 4 – Configuring an Object
Since our task here is to define the Datastreams in the object, click on the Datastreams tab and you will see a window like that shown in Figure 5. Note that at this point there is only one Datastream in the object – the DC Datastream containing basic descriptive metadata that was automatically created by Fedora. You can select that Datastream and select the Edit
button to see the its default contents, with the DC title and identifier fields already filled in.
Figure 5 – Datastream Display
A few points to note about what you have done so far:
Control Group
of the DC Datastream is Internal XML Metadata
. As explained earlier, Fedora has a number of control group types, of which this is one. This type is appropriate for metadata that is represented in XML – Dublin core metadata being one example. A digital object can have multiple metadata Datastreams, for example MARC, LOM, Dublin Core, and others. Edit
button and modifying the contents of the text pane. When you press Save Changes...
, Fedora will check that the Datastream is well-formed XML.You may also create Dublin Core metadata (or any other XML-based metadata) in an external XML editor and using the Import...
button to replace the Datastream with this data. When you press Save Changes...
, Fedora will check that the Datastream is well-formed XML.
You will notice that there are optional fields on the Datastreams pane for Format URI
(to refine the media type meaning with a URI that more precisely identifies the media type) and Alternate Ids
to capture any other existing identifiers you would like to associate with a Datastream. We will not be using these in this tutorial.
It is now time to add the ePrints document formats as new Datastreams. You can find content for creating the Datastreams for this example in:
FEDORA_HOME/userdocs/tutorials/2/example1/artex.html
FEDORA_HOME/userdocs/tutorials/2/example1/artex.pdf
FEDORA_HOME/userdocs/tutorials/2/example1/artex.tex
To do this, select the New...
tab on the left side of the Datastreams window. We'll start with the text/html
format. To insert data into the Datastream, you use the Import...
button. This presents a dialog that will allow you to import from your local file system or from a URL.
Your completed HTML Datastream should look like the dialog as shown in Figure 6 (after you have imported the content).
Figure 6 – Adding a New Managed Content Datastream
A few notes on the contents of this dialog:
ID
of the Datastream should be a single token. By convention, it describes the purpose of the Datastream.Label
can be a longer, more descriptive string.Control Group
is Managed Content
. As shown in the descriptive text this Datastream type is appropriate for any type of data (mime type), in contrast to Internal XML Metadata
. Once you select this radio button, you can select from the variety of MIME Types
of the managed content – in this case text/html
.You can now select the Save Datastream
button and repeat the same process to add the pdf
and TeX
Datastreams. For the pdf
, you can select Mime Type: application/pdf
and import the file ex1.pdf
. For TeX, you can select Mime Type: text/plain
and import the file ex1.tex
. In each case you should enter appropriate IDs and Labels.
You're done! Your Datastreams window should now look something like that shown in Figure 7, showing all the Datastreams you have entered in the left-side tabs in the window.
Figure 7 – Complete Datastreams for Example 1
You will notice as you click through each Datastream that there is a Fedora URL, giving the unique URL to access each Datastream from the Fedora repository. Try going to a browser and entering one of these URLs – the browser will download the Datastream and display it. These URLs can be used by web applications and REST-based web services that access Datastreams from Fedora digital objects. Note that if you are building SOAP-based web services, there are also SOAP methods (getDataStream
and getDatastreamDissemination
) that provide Datastream access. You can also try entering the root URL for the entire digital object, which is simply the common prefix of all the Datastream URLs – e.g., http://localhost:8080/fedora/get/demo:100. This accesses the header page for the digital object, which allows you to access its Datastreams (available through the item index hyperlink) and disseminations (available through the dissemination index hyperlink).
Figure 8 illustrates the structure of the object you have created and the correspondence of REST-based access requests to the object and its components (via API-A-LITE).
Figure 8 – Example 1 Digital Object and Datastreams
The previous example demonstrated how to aggregate imported content into a Fedora digital object. There are many reasons why importing content into a repository might not be appropriate such as rights restrictions or the dynamic nature of the content. To accommodate these restrictions, digital objects in Fedora may contain Datastreams that reference externally managed content, and in fact may mix local and distributed data sources.
This section describes how to do this where the motivating example is the creation of a hypothetical learning object in an educational digital library, such as the NSDL}. The digital object created in this example combines three frog images from the NSDL collection and some locally-managed text.
To get started follow the same procedure as illustrated in Figure 3, this time entering Example 2 as the Label and demo:200 as the custom PID. As in Example 1, select the Datastreams tab and then enter the information as shown in Figure 9.
Figure 9 – Adding a Datastream with Type Redirect
You will enter the Datastream identifier of IMAGE1, a label for this Datastream, and then information about the content. The content is of MIME type image/gif
. You should select the Control Group of Redirect, and then enter a URL that specifies the Location of the image file, specifically:
http://www.frogsonice.com/froggy/images/toads.gif
A few notes on the contents of this dialog:
In the same manner, you can now proceed to add the two other datastreams with locations: http://www.werc.usgs.gov/fieldguide/images/hycafr.jpg and http://www.aquariumofpacific.org/images/olc/treefrog600.jpg
You should respectively identify these datastreams as IMAGE2 and IMAGE3. (Note that if these sample URLs are no longer active, you can enter other URLs pointing to jpeg images to complete this tutorial exercise.)
Finally, add another datastream labeled MyText (containing some descriptive text about the images), with MIME Type text/html. Assign this datastream a Control Group of Managed Content indicating that the content will be imported and stored permanently in the Fedora repository. Import the content from the following location:
FEDORA_HOME/userdocs/tutorials/2/example2/mytext.html.
The resulting datastream window should now look like that shown in Figure 10.
Figure 10 – Example 2 Datastream Display
You're done! Figure 11 illustrates the role of the redirected Datastream at the time of digital object access via the Fedora REST-based interface (API-A-LITE). You can see this by going to the digital object profile page at: http://localhost:8080/fedora/get/demo:200
You can access the Datastreams for this digital object by viewing the item linked to from the object profile page. Then, select the link for one of the redirected Datastreams. Fedora will redirect your browser to the location of the datastream content, without streaming the content through the Fedora repository server.
Figure 11 – Example Digital Object and Redirected Datastream
The examples described so far demonstrate the basic content aggregation features of Fedora. As mentioned already, the power of Fedora lies in its ability to associate the data in a digital object with web services to produce dynamic disseminations.Some examples of this capability are as follows:
The remainder of this section presents a series of examples demonstrating how to create digital objects that exploit web services. The initial examples make use of services available in the Fedora software release (they run as "local services" within the Fedora server container). Later examples demonstrate how to construct your own custom objects with external web services. Before proceeding with the examples, this introduction summarizes the concepts and defines the terms used in the examples. Don't worry if the concepts are not entirely clear at first. You should read them now and then refer back to them as you work through the examples.
Figure 12 shows an abstract view of the different components of the Fedora repository architecture that are key to how Fedora produces "disseminations" of digital object content.
Figure 12 – Abstract View: Key Fedora Components for Producing Disseminations of Content
These layers are:
The process of creating digital objects with dynamic content disseminations involves creating linkages between these layers. During this process you will create and employ the following:
These three kinds of special Fedora objects are stored in Fedora repositories. The set of all SDefs represents a "registry" of all the kinds of abstract services supported by the Fedora repository. The set of all SDeps represents a "registry" of all the concrete service bindings for the abstract service definitions supported by the Fedora repository. The set of all CModels represent a "registry" of the different user-defined types of data objects that exist in that Fedora repository.
At the end of the day, digital objects make references to SDefs, SDeps and CModels as the way of providing extended access points for digital objects (i.e., dynamic content disseminations.) This is done by adding special relationships between the objects that are stored in the RELS-EXT datastreams of those objects.
Figure 13 indicates the relationships that exist between the four object types. Data objects assert that they conform to a particular Content Model using the hasModel relationship. Content Model objects assert they provide the services included in an SDef using the hasService relationship. Service Deployment objects assert the services for which they provide binding information by using the isDeploymentOf relationship, as well as asserting the Content Models for which they provide service bindings using the isContractorOf relationship.
Figure 13 – Fundamental CMA Relationships
Figure 14 illustrates the interactions among Fedora and Web services in response to an access request. As indicated, a client makes a request to the Fedora API (with a URL in this case.) The Fedora repository service then determines the content model that is associated with the digital object for which the request is being made. Once it knows the content model, the Fedora repository can discover what SDefs and SDeps are in play for this digital object. Once all of this information is gathered, the Fedora repository can construct a request to the appropriate web service to transform the datastreams of the target digital object (demo:2). The Fedora repository service invokes a REST-based request to the web service via HTTP, sending along arguments to enable the web service to obtain the required datastream inputs to fulfill the request. The Fedora repository mediates all invocations with the external web service. When it receives a response from the web service it streams it back to the original calling client. In this case, the response is a transformation based on the raw material of Datastream1 and Datastream2 in the digital object.
Figure 14 – Dynamic Dissemination Access
This example makes use of a SDef, SDep and CModel supplied with the Fedora tutorial. This will help you understand the basics of dynamic disseminations in Fedora under the Content Model Architecture, without writing a SDef, SDep or CModel. The next example describes how to do that more advanced task.
The web service used in the example performs an XSLT transform using the well-known saxon XSLT processor. This service requires two inputs, an XML source document and a XSL transform document. In this example, both of these XML documents are stored as managed content in a Fedora digital object. The XML source is data for a poem with tags for the structural elements of the poem (stanzas and lines). The XSL transform produces a HTML output of the poem that can be viewed in a browser. This example is borrowed from the web available source for Michael Kay's excellent XSLT book.
First we will ingest a sample SDef object into the repository.
Select File/Ingest/One Object/From File... in the Fedora Administrator. This will bring up a file selection dialog box as follows:
Browse the file system to select the ingest file for the SDef object whose file name is FEDORA_HOME/userdocs/tutorials/2/example3/SDef.xml
. Since this ingest file is encoded as FOXML 1.1 select the FOXML 1.1 radio button as below:
This will create the digital object with PID demo:ex3SDef in your repository. This SDef defines one method getContent. This generic method name is intentional – one could imagine this one SDef being used as the basis for several SDeps, each of which produces "content" via a unique transformation of an underlying source. This is one of the advantages of Fedora – providing a common interface despite multiple underlying representations.
Follow the same procedure to ingest a sample SDep object into the repository. Select the file FEDORA_HOME/userdocs/tutorials/2/example3/SDep.xml. This will create the digital object with the PID demo:ex3SDep. This SDep represents a concrete implementation of the abstract service operations defined in the SDef demo:ex3SDef . The SDep object contains metadata that specifies the following:
Next follow the same procedure to ingest a sample CModel object into the repository. Select the file FEDORA_HOME/userdocs/tutorials/2/example3/CModel.xml. This will create the digital object with the PID demo:ex3CModel. This CModel describes the datastreams that should be present in data objects that conform to this content model, it also has a RELS-EXT hasService relationship link to the digital object demo:ex3SDef ingested previously.
Now you need to create the new digital object based on this SDef, SDep and CModel. To get started follow the same procedure as illustrated in Figure 3, this time entering demo:300 as the datastream ID and Example 3 as the Label.
You now need to add the two datastreams: the xml source document and the xsl transform document. Using the same method described in Example 1, select the Datastreams tab and:
In Fedora Administrator, select the Datastreams tab from the digital object window, and then select the New RELS-EXT... tab. The resulting dialog will now allow you to create the necessary RELS-EXT relationship to allow dynamic dissemination to work. Follow these steps:
The resulting Object window should look like that illustrated in Figure 15.
Figure 15 – Example 3 Linking a Digital Object to a Content Model
You're done! Figure 16 illustrates the role of this digital object and disseminator in response to a client request. You can go to the digital object header page at http://localhost:8080/fedora/get/demo:300 and select the View Dissemination Index link. Your newly added dynamic dissemination should now appear, alongside the primitive behaviors for the object. To see the results of this dynamic dissemination, you can either select the Run button for getContent in the Method Index display or simply enter the URL http://localhost:8080/fedora/get/demo:300/demo:ex3SDef/getContent directly.
Figure 16 – Example 3 dissemination via CMA
Example 3 packages the XSL transform datastream in the same digital object as the source XML datastream. However, in many cases you will have XSL transform code that you want to share across several XML sources. This section modifies Example 3 to enable this sharing.
This is done by packaging the XSL transform code in a digital object of its own. Then every digital object that needs to make use of the XSL transform code can use the Fedora REST URL to access that datastream. This is done by defining a redirect datastream using the REST URL as the redirect target. Then, the same disseminator design used in Example 3 can be reused. This is known as dissemination chaining, whereby the dissemination of one digital object is used by another.
The steps to do this are quite simple and use techniques introduced thus far:
You're done! The demo:500 digital object should now behave exactly the same as the demo:300 digital object in Example 3. Figure 17 refines Figure 16 (with some labeling removed for clarity) with the new redirect configuration.
Figure 17 - Dissemination with redirect datastream
You should now understand the basic mechanisms through which SDefs, SDeps and CModels interact with Data objects to provide a richer dynamic view of the data stored in those objects. The next tutorial (Tutorial 3 – Not yet available) steps you through the process of using the admin client to create a SDef, a SDep, and a CModel from scratch and a Data object that will function with the control objects to provide customized services similar to those described in the last example of this tutorial. To explore the other features of Fedora, refer to the Fedora Repository Documentation. You can also join the Fedora-users mailing list to ask questions and learn from the experience of other Fedora users.