...

{center}
*Fedora Tutorial #2* 
*Getting Started:*   
*Creating Fedora Objects using the*  
*Content Model Architecture*
*Fedora 3.0* \\ \\
July 23, 2008 \\ \\
{center}

Author: The Fedora Development Team

...

Figure 1 - Fedora repository as mediator for services and content
Figure 2 - Fedora Administrator Login Screen
Figure 3 - New object dialog
Figure 4 - Configuring an object
Figure 5 - Datastream display
Figure 6 - Adding a new managed content datastream
Figure 7 - Complete datastreams for example 1
Figure 8 - Example 1 digital object and datastreams
Figure 9 - Adding a datastream with type Redirect
Figure 10 - Example 2 datastream display
Figure 11 - Example digital object and redirected datastream
Figure 12 - Abstract View: Key Fedora Components for Producing Disseminations of Content
Figure 13 - Relationships between Data objects and CModel/SDef/SDep objects for CMA
Figure 14 - Dynamic dissemination access
Figure 15 - Example 3 Linking a Digital Object to a Content Model
Figure 16 - Example 3 dissemination via CMA
Figure 17 - Dissemination with redirect datastream

What is this document and who should read it?

This is an introduction for system developers and repository managers who are new to the Fedora open-source content management software. This is a hands-on tutorial. It assumes that you have already installed the Fedora software and are at a computer with access to a Fedora repository through the Fedora Administrator while reading this tutorial.
You don't have to have to be a programmer to understand and use this tutorial. However, you should be familiar with the operation and structure of web servers and web services.
This document is not intended for end users of content disseminated by a Fedora repository.

What is Fedora and what does it do?

Fedora is content management software that runs as a web service within an Apache Tomcat web server. Fedora provides the tools and interfaces for creation, ingest, management, and dissemination of content stored within a repository. There are a number of features that distinguish Fedora:

1. It supports the creation and management of digital content objects (from this point on called digital objects) that can aggregate data from multiple sources. For example, a digital object might be a set of tiff images that are the individual page images of a scanned document. The data sources may be either locally managed within the Fedora software or sourced from another URL accessible network server. The data sources may be content or metadata. You may think of these digital objects as advanced digital documents, especially in light of the feature described next.

...

1. It supports the association of web services with the digital objects. These services typically consume the data packaged within the digital object to produce dynamic disseminations from the digital object. For example, the digital object described above with multiple tiff page images may be associated with a service that OCRs the images that are components of the digital object and disseminates an html version of the pages. The services may be either local to the machine of the respective Fedora server or sourced from another network accessible server that is addressable via a URL. In this manner, Fedora acts as a mediation layer that coordinates local and distributed data and web services within a uniform framework. This is illustrated in Figure 1.

...

1. It provides uniform access web-based interfaces to these digital objects, through REST requests and more powerful SOAP-based methods. These interfaces consist of a set of built-in methods to access characteristics common to all digital objects such as key metadata and internal structure. These include a method to introspect on an object to reveal the set of methods that constitute the extended behavior of that object. For example, a client could use these built-in methods to "learn" about the capability of the digital object described above to dynamically disseminate an html page from a set of tiff images.
   The benefits of these are two-fold:

   ...

   1. 1. Clients accessing Fedora digital objects can rely on uniform access regardless of the nature of the object.

   ...

   1. 1. The disseminations available from an object are independent of the internal structure of the object. For example, the client interface of the example above in which html is disseminated from a set of source tiff pages could remain constant regardless of whether the underlying object contained tiff images, jpeg, pdf, or even simple static html. This gives the content developer great freedom to modify a repository's internals without disrupting the client and user views of the content.
   2. It presents a uniform and powerful SOAP-based management interface. All internal operations of the repository such as object creation and management are available through this API, providing the hooks for integrating Fedora into a variety of environments. These makes Fedora useful as the foundation for advanced content management applications
   3. It includes a comprehensive versioning framework that tracks the evolution of objects and provides access to earlier versions.
   4. It includes a basic relationship framework for representing the links among digital objects.
   5. It supports ingest and export of digital objects in a variety of XML formats. This enables interchange between Fedora and other XML-based applications and facilitates archiving tasks.

   A number of these features are illustrated in Figure 1.

   
   Figure 1 - Fedora repository as mediator for services and content

   Why should you use Fedora?

   Fedora may be the wrong choice for management of simple static web pages. There are a number of excellent tools for html editing and web site creation. Fedora is more appropriate for more advanced content management tasks. These include management of content and associated metadata, multiple versions of content, content available in multiple formats, and dynamically generated content from local and dynamic sources.

   How should you read this document?

   This document is intended to be hands-on, with you trying the examples on a running Fedora repository. You should therefore, have already downloaded and installed Fedora, and started a server. You should then access the Fedora repository by running the Fedora Administrator interface, fedora-admin, which is located in the FEDORA_HOME/client directory (you can start this program from the command line if you have configured your environment variables properly). Upon starting up the administrator interface you will be presented with the login screen shown in Figure 2. This document assumes that you have not changed any of the configuration defaults for your Fedora server so the Password you enter should be fedoraAdmin. If you have changed your configuration values or are running the Fedora Administrator from a machine different from the machine on which your Fedora server is running you will need to change the values in the Login screen appropriately.
   
   
   Figure 2 - Fedora Administrator Login Screen
   You should read this document in order, since later examples assume knowledge of techniques and definitions introduced earlier.

   Conventions used in this document

   The font conventions used are:

   ...

   All pathnames assume that you have set your FEDORA_HOME environment variable and descend from the directory defined by that variable.

   All URLs that access the Fedora repository assume that the host:port of the repository is localhost:8080.

   Getting Started: Using Fedora for Aggregating Content

   This section describes how to create digital objects in Fedora that aggregate data from multiple sources. The examples demonstrate how to do this with both local data and data from networked sources. This section provides the foundation for the next section, which describes how to use Fedora to create dynamic content by exploiting web services. Make sure you understand the basic concepts here, before moving on to that next section

   ...

   Some basic definitions_

   To understand content aggregation in Fedora, you need to be comfortable with two terms:
   1. Digital Object - This is the basic unit for information aggregation in Fedora. At a minimum a digital object has:
   a. An identifier or PID (Persistent Identifier). This PID provides the key by which the digital object is accessed from the repository.
   b. Dublin Core metadata that provides a basic description of the digital object.
   2. Datastream - A component of a digital object that represents a data source. A digital object may have just the basic Dublin Core datastream, or any number of additional datastreams. Each datastream can be any mime-typed data or metadata, and can either be content managed locally in the Fedora repository or by some external data source (and referenced by a URL). When you create a new datastream in a digital object, you assign it to one of four types, or control groups, depending on the nature of the data that it represents.
   a. Managed Content (M): Datastream content is stored and managed within the Fedora repository's persistent storage. The content can be any MIME type including XML.
   b. Inline XML (X): A special case of M, restricted to well-formed XML. In this case the datastream content is stored as part of the XML structure of the digital object itself and is thus included when the digital object is exported (e.g., for archival purposes).
   c. Externally Referenced (E): Datastream content is external to the Fedora repository and is referenced by a URL that is recorded within the digital object. The content can be any MIME type including XML.
   d. Redirected Content (R): Like E, but datastream content is delivered to the client without any mediation by Fedora; i.e., via an HTTP redirect. You should use this datastream type when the external content is a web page with relative links or it is streaming audio or video. The content can be any MIME type including XML.
   Decisions about what to include in a digital object and how to configure its datastreams are basic modeling choices as you develop your repository. The examples in this tutorial demonstrate some common models that you may find useful as you develop your application.

   ...

   Example 1: Making a document available in multiple formats_

   It is often useful to provide access to a digital document in several formats. For example an ePrints server might provide html for those who wish to render the document in a browser, pdf for those who wish to view the document with author-determined formatting, and TeX for those who wish to access and use the document source. This example demonstrates how to construct a digital object where each datastream corresponds to an available format. More advanced techniques, demonstrated later in this tutorial, make it possible to achieve the same results by generating formats dynamically from a single base format. But for now, we'll stick to simple static aggregation.
   Start by selecting File/New/Data Object in the Admin GUI. Complete the New Object dialog box as shown in Figure 3.
   
   
   Figure 3 - New object dialog
   Check the box for Use Custom PID and enter demo:100. Note that when you do not assign your own PID, the Fedora repository will create one for you. Select the Create button and you should see a window like that in Figure 4. Observe that the PID of the created object (in this case demo:100) is displayed in the title bar.
   
   
   Figure 4 - Configuring an object
   
   
   Since our task here is to define the datastreams in the object, click on the Datastreams tab and you will see a window like that in Figure 5. Note that at this point there is only one datastream in the object - the DC datastream for basic descriptive metadata that was automatically created by Fedora. You can select that datastream and select the Edit button to see the default contents of this Datastream, with the DC title and identifier fields already filled in.
   
   
   Figure 5 - Datastream display
   
   A few points to note about what you have done so far:

   ...

   
   You can now select the Save Datastream button and repeat the same process to add the pdf and TeX datastreams. For the pdf, you can select Mime Type: application/pdf and import the file ex1.pdf. For TeX, you can select Mime Type: text/plain and import the file ex1.tex. In each case you should enter appropriate IDs and Labels.
   You're done! Your Datastreams window should now look something like that shown in Figure 7, showing all the datastreams you have entered in the left-side tabs in the window.
   
   
   Figure 7 - Complete datastreams for example 1
   You will notice as you click through each datastream that there is a Fedora URL, giving the unique URL to access each datastream from the Fedora repository. Try going to a browser and entering one of these URLs - the browser will download the datastream and display it. These URLs can be used by web applications and REST-based web services that access datastreams from Fedora digital objects. Note that if you are building SOAP-based web services, there are also SOAP methods (getDataStream and getDatastreamDissemination) that provide datastream access. You can also try entering the root URL for the entire digital object, which is simply the common prefix of all the datastream URLs - e.g., http://localhost:8080/fedora/get/demo:100. This accesses the header page for the digital object, which allows you to access its datastreams (available through the item index hyperlink) and disseminations (available through the dissemination index hyperlink).
   Figure 8 illustrates the structure of the object you have created and the correspondence of REST-based access requests to the object and its components (via API-A-LITE).
   
   
   Figure 8 - Example 1 digital object and datastreams

   ...

   Example 2: Creating a surrogate for distributed content_

   The previous example demonstrated how to aggregate imported content into a Fedora digital object. There are many reasons why importing content into a repository might not be appropriate such as rights restrictions or the dynamic nature of the content. To accommodate these restrictions, digital objects in Fedora may contain datastreams that reference externally managed content, and in fact may mix local and distributed data sources.
   This section describes how to do this where the motivating example is the creation of a hypothetical learning object in an educational digital library, such as the NSDL. The digital object created in this example combines three frog images from the NSDL collection and some locally-managed text.
   
   To get started follow the same procedure as illustrated in Figure 3, this time entering Example 2 as the Label and demo:200 as the custom PID. As in Example 1, select the Datastreams tab and then enter the information as shown in Figure 9.
   
   Figure 9 - Adding a datastream with type Redirect
   You will enter the datastream identifier of IMAGE1, a label for this datastream, and then information about the content. The content is of MIME type image/gif. You should select the Control Group of Redirect, and then enter a URL that specifies the Location of the image file, specifically:
   http://www.frogsonice.com/froggy/images/toads.gif
   A few notes on the contents of this dialog:
   • Pertaining to the selection of a Control Group, you have two choices if you want the datastream to point to content that resides outside the Fedora repository (External Referenced Content and Redirect). In this case we chose Redirect . To review, the meaning of the two options for mapping to external content are:
   o External Referenced Content is useful when you want Fedora to mediate access to the datastream, for example when you want to hide the source URL from the user. Fedora mediates access to these datastreams, meaning that the content is streamed through the Fedora server.
   o Redirect. makes use of a simple HTTP redirect to provide the content. This is useful when there are relative hyperlinks in the external content, but reveals the source URL to the user.
   • Make sure that the MIME Type choice matches that of the content offered by the external source, in this case image/gif.
   In the same manner, you can now proceed to add the two other datastreams with locations: http://www.werc.usgs.gov/fieldguide/images/hycafr.jpg and http://www.aquariumofpacific.org/images/olc/treefrog600.jpg
   You should respectively identify these datastreams as IMAGE2 and IMAGE3. (Note that if these sample URLs are no longer active, you can enter other URLs pointing to jpeg images to complete this tutorial exercise.)
   Finally, add another datastream labeled MyText (containing some descriptive text about the images), with MIME Type text/html. Assign this datastream a Control Group of Managed Content indicating that the content will be imported and stored permanently in the Fedora repository. Import the content from the following location:
   FEDORA_HOME/userdocs/tutorials/2/example2/mytext.html.
   The resulting datastream window should now look like that shown in Figure 10.
   
   
   Figure 10 - Example 2 datastream display
   You're done! Figure 11 illustrates the role of the redirected datastream at the time of digital object access via the Fedora REST-based interface (API-A-LITE). You can see this by going to the digital object profile page at: http://localhost:8080/fedora/get/demo:200
   You can access the datastreams for this digital object by viewing the item linked to from the object profile page. Then, select the link for one of the redirected datastreams. Fedora will redirect your browser to the location of the datastream content, without streaming the content through the Fedora repository server.
   
   
   Figure 11 - Example digital object and redirected datastream

   Using Fedora to produce dynamic content

   The examples described so far demonstrate the basic content aggregation features of Fedora. As mentioned already, the power of Fedora lies in its ability to associate the data in a digital object with web services to produce dynamic disseminations. Some examples of this capability are as follows:

   ...

   
   These three kinds of special Fedora objects are stored in Fedora repositories. The set of all SDefs represents a "registry" of all the kinds of abstract services supported by the Fedora repository. The set of all SDeps represents a "registry" of all the concrete service bindings for the abstract service definitions supported by the Fedora repository. The set of all CModels represent a "registry" of the different user-defined types of data objects that exist in that Fedora repository.
   At the end of the day, digital objects make references to SDefs, SDeps and CModels as the way of providing extended access points for digital objects (i.e., dynamic content disseminations.) This is done by adding special relationships between the objects that are stored in the RELS-EXT datastreams of those objects.
   Figure 13 indicates the relationships that exist between the four object types. Data objects assert that they conform to a particular Content Model using the hasModel relationship. Content Model objects assert they provide the services included in an SDef using the hasService relationship. Service Deployment objects assert the services for which they provide binding information by using the isDeploymentOf relationship, as well as asserting the Content Models for which they provide service bindings using the isContractorOf relationship.
   
   Figure 13 - Fundamental CMA Relationships
   
   
   Figure 14 illustrates the interactions among Fedora and Web services in response to an access request. As indicated, a client makes a request to the Fedora API (with a URL in this case.) The Fedora repository service then determines the content model that is associated with the digital object for which the request is being made. Once it knows the content model, the Fedora repository can discover what SDefs and SDeps are in play for this digital object. Once all of this information is gathered, the Fedora repository can construct a request to the appropriate web service to transform the datastreams of the target digital object (demo:2). The Fedora repository service invokes a REST-based request to the web service via HTTP, sending along arguments to enable the web service to obtain the required datastream inputs to fulfill the request. The Fedora repository mediates all invocations with the external web service. When it receives a response from the web service it streams it back to the original calling client. In this case, the response is a transformation based on the raw material of Datastream1 and Datastream2 in the digital object.
   
   
   Figure 14 - Dynamic dissemination access

   ...

   Example 3: Using SDefs, SDeps and CModels_

   This example makes use of a SDef, SDep and CModel supplied with the Fedora tutorial. This will help you understand the basics of dynamic disseminations in Fedora under the Content Model Architecture, without writing a SDef, SDep or CModel. The next example describes how to do that more advanced task.

   The web service used in the example performs an XSLT transform using the well-known saxon XSLT processor. This service requires two inputs, an XML source document and a XSL transform document. In this example, both of these XML documents are stored as managed content in a Fedora digital object. The XML source is data for a poem with tags for the structural elements of the poem (stanzas and lines). The XSL transform produces a HTML output of the poem that can be viewed in a browser. This example is borrowed from the web available source for Michael Kay's excellent XSLT book.

   ...

   Ingesting pre-defined SDef, SDep and CModel objects

   First we will ingest a sample SDef object into the repository.

   Select File/Ingest/One Object/From File... in the Fedora Administrator. This will bring up a file selection dialog box as follows:

   

   Browse the file system to select the ingest file for the SDef object whose file name is FEDORA_HOME/userdocs/tutorials/2/example3/SDef.xml. Since this ingest file is encoded as FOXML 1.1 select the FOXML 1.1 radio button as below:

   

   This will create the digital object with PID demo:ex3SDef in your repository. This SDef defines one method getContent. This generic method name is intentional - one could imagine this one SDef being used as the basis for several SDeps, each of which produces "content" via a unique transformation of an underlying source. This is one of the advantages of Fedora - providing a common interface despite multiple underlying representations.
   Follow the same procedure to ingest a sample SDep object into the repository. Select the file FEDORA_HOME/userdocs/tutorials/2/example3/SDep.xml. This will create the digital object with the PID demo:ex3SDep. This SDep represents a concrete implementation of the abstract service operations defined in the SDef demo:ex3SDef . The SDep object contains metadata that specifies the following:
   • Service Contract: the SDep indicates the PID of the SDef that it is related to. This is like saying that the SDep provides and implementation of the SDef.
   • Service binding metadata (i.e., in WSDL) : concrete binding for the getContent method that is defined. Specifically, the WSDL indicates that the getContent operation binding exists at the base URL of http://localhost:8080/service/saxon. Note that this service is hosted at the same host and port as the Fedora repository. As noted earlier, this is a local service that is packaged with Fedora.
   • Data input profile that indicates that the SDep service operation getContent will take the following inputs at runtime:
   o "xsl" with MIME type text/xml.
   o "source" with MIME type text/xml.
   Next follow the same procedure to ingest a sample CModel object into the repository. Select the file FEDORA_HOME/userdocs/tutorials/2/example3/CModel.xml. This will create the digital object with the PID demo:ex3CModel. This CModel describes the datastreams that should be present in data objects that conform to this content model, it also has a RELS-EXT hasService relationship link to the digital object demo:ex3SDef ingested previously.
   

   ...

   
   

   Creating a digital object with appropriate datastreams

   
   
   Now you need to create the new digital object based on this SDef, SDep and CModel. To get started follow the same procedure as illustrated in Figure 3, this time entering demo:300 as the datastream ID and Example 3 as the Label.
   You now need to add the two datastreams: the xml source document and the xsl transform document. Using the same method described in Example 1, select the Datastreams tab and:
   • Add a datastream with:
   o ID - source
   o Control Group - Managed Content
   o Mime type - text/xml
   o Label - Poem XML Source
   o Import location: FEDORA_HOME/userdocs/tutorials/2/example3/poem.xml
   • Add a datastream with:
   o ID - xsl
   o Control Group - Managed Content
   o Mime type - text/xml
   o Label - Poem XSL Transform
   o Import location: FEDORA_HOME/userdocs/tutorials/2/example3/poem.xsl

   ...

   Linking the digital object to the Content Model

   In Fedora Administrator, select the Datastreams tab from the digital object window, and then select the New RELS-EXT... tab. The resulting dialog will now allow you to create the necessary RELS-EXT relationship to allow dynamic dissemination to work. Follow these steps:

   ...

   
   
   The resulting Object window should look like that illustrated in Figure 15.
   
   
   Figure 15 - Example 3 Linking a Digital Object to a Content Model
   You're done! Figure 16 illustrates the role of this digital object and disseminator in response to a client request. You can go to the digital object header page at http://localhost:8080/fedora/get/demo:300 and select the View Dissemination Index link. Your newly added dynamic dissemination should now appear, alongside the primitive behaviors for the object. To see the results of this dynamic dissemination, you can either select the Run button for getContent in the Method Index display or simply enter the URL http://localhost:8080/fedora/get/demo:300/demo:ex3SDef/getContent directly.

   
   Figure 16 - Example 3 dissemination via CMA

   ...

   * Example 4 - Modifying Example 3 using a redirect datastream_*

   Example 3 packages the XSL transform datastream in the same digital object as the source XML datastream. However, in many cases you will have XSL transform code that you want to share across several XML sources. This section modifies Example 3 to enable this sharing.
   This is done by packaging the XSL transform code in a digital object of its own. Then every digital object that needs to make use of the XSL transform code can use the Fedora REST URL to access that datastream. This is done by defining a redirect datastream using the REST URL as the redirect target. Then, the same disseminator design used in Example 3 can be reused. This is known as dissemination chaining, whereby the dissemination of one digital object is used by another.
   The steps to do this are quite simple and use techniques introduced thus far:
   • Create a new digital object (the "XSL" digital object) assigning the PID demo:400. Create one datastream in addition to the DC with ID XSL. As before, this datastream should be configured as:
   o ID - xsl
   o Control Group - Managed Content
   o Mime type - text/xml
   o Label - Poem XSL Transform
   o Import location: FEDORA_HOME/userdocs/tutorials/2/example3/poem.xsl
   • Create another digital object (the "disseminator" digital object) assigning the PID demo:500.
   • Create two new datastreams
   o One configured as follows (the same as the Source datastream in Example 3):

   ...