This documentation space is deprecated. Please make all updates to DuraCloud documentation on the live DuraCloud documentation space.

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 32 Next »

Introduction

The DuraCloud application provides a set of services which can be deployed and used for a variety of purposes, primarily to process the content which has been loaded into DuraCloud storage. The following list of services describes how each service is expected to be used and the options available for tailoring the service to your needs.

If you start a service, you will receive an email when it completes processing. For each of the services below, if they are run independently, they will not auto-restart if they fail during processing. If you are made aware of a failed job state, you have the option to redeploy the service. Some services are run automatically by the DuraCloud Executor, and as such their state is managed by the Executor. In particular, these are the Media Streamer and the Bit Integrity Tools services.

Not all services are available in all service plans.

Duplicate on Change

Description:

The Duplicate on Change service provides a way to ensure that the content stored in DuraCloud is synchronized between different storage providers. The Duplicate on Change service duplicates any changes made to spaces, content, or properties for the spaces it is configured to watch. This means that once the Duplicate on Change service is deployed, it notices all content that is added, updated, or deleted for each configured space in the watched DuraCloud provider and performs the same functions on the selected secondary provider. All content that is copied will be placed in an identically named space in the secondary storage location with the same property fields attached. The duplication provided by this service is one-way; only the provider that is selected to be watched is monitored for changes. There is also an option to set a default configuration, which is then applied to any space that is added after the Duplicate on Change service has started. This ensures that content added to new spaces is duplicated properly.

Note that this service only performs duplication of content after it has been deployed. Content that exists in the provider prior to this service being deployed will not be duplicated. Existing content can be copied into a space on the primary provider (after the Duplicate on Change has been deployed to watch it) in order to ensure it is duplicated to the secondary provider.

Configuration Options:

  1. Store to Watch: The primary storage location which DuraCloud will monitor for changes. When spaces, content, or properties are added, updated, or deleted in this store, the same actions will be taken for the configured spaces in a secondary store.
  2. Space to store selection: Each space in the watched provider will be duplicated in the storage provider(s) selected. For each space, 0, 1, or more providers may be selected.
  3. Default space to store selection: Any new space added will be configured based on this default setting. This works the same as the space to store setting above, but applies only to spaces added after the Duplicate on Change service is deployed.

Media Streamer

Description:

The Media Streamer provides streaming capabilities for video and audio files. The service takes advantage of Amazon Cloudfront streaming, so files to be streamed must be within spaces on an Amazon provider.

Amazon Cloudfront streaming uses the Flash Media Server to host streaming files over RTMP. File formats supported include MP3, MP4 and FLV among others. For a full listing of supported file types see the Flash Media Server documentation.

Configuration Options:

  1. Source Media Space: The DuraCloud space(s) where the source video and audio files to be streamed are stored. The Media Streamer service attempts to stream all files in the selected space(s).

Integration Files
The following files are available as a bundle on the downloads page.
They are intended as a starting point for integrating streaming media into your own website.

  • player.swf - The flash-based video player JWPlayer
  • playlist.xml - An example playlist which would include a list of items in your Source Media Space
  • playlistplayer.html - An HTML file which uses JWPlayer to display the items in the playlist
  • singleplayer.html - An HTML file which uses JWPlayer to display a single media file
  • stylish.swf - A supplementary flash file used to style the JWPlayer
  • swfobject.js - A javascript file (available from here) used to embed the JWPlayer on a web page
  • viewer.js - A javascript file used to simplify the loading of JWPlayer

All of the above files are intended as examples only. Their purpose is give developers a starting point for embedding video streamed by DuraCloud on their own web pages.

If you add files when the Media Streamer service is already running, they too will automatically be available for streaming.

Bit Integrity Checker

Description:

The Bit Integrity Checker provides the ability to verify that the content held within DuraCloud has maintained its bit integrity. There are two modes of operation.
Modes:

  1. Verify integrity of a Space
  2. Verify integrity of an item list

When running in the Verify integrity of a Space mode, the checker performs the following steps

  • collect the content hash values for each item from the underlying storage provider
  • stream through each item recalculating their hashes
  • compare the two listings

When running in the Verify integrity of an item list mode, the checker performs the following steps

  • stream through each item in the provided listing, recalculating their hashes
  • compare the newly generated listing with the provided listing

Configuration Options:

  1. Stores: The underlying storage provider over which the service will run
  2. Space containing content items: The DuraCloud space in which the content items to be verified reside
  3. Verify integrity of an item list mode
    1. Input listing name: Name of the content item which contains the listing of items over which to run the service

Service Ouputs
All outputs of this service are placed in the system space, x-service-out.

  1. bitintegrity/fingerprints-gen-<spaceId>-<date>.csv
    • Interim listing generated with hash values from underlying storage provider
  2. bitintegrity/fingerprints-<spaceId>-<date>.csv
    • Interim listing with hashes recalculated from content streams
  3. bitintegrity/fixity-report-<spaceId>-<date>.csv
    • Final report with status of integrity check

Bit Integrity Checker - Tools

Description:

The Bit Integrity Checker Tools provide additional bit integrity checking utilities which can be used to perform specific integrity checking tasks.

Modes:

  1. Generate integrity information for a Space
  2. Generate integrity information for an item list
  3. Compare two integrity reports

Configuration Options:

  1. Mode 1 - Generate integrity information for a Space
    1. Get integrity information from...
      1. The storage provider: Determine the file MD5 by asking the storage provider for its stored MD5 value
      2. The files themselves: Determine the file MD5 by retrieving them from the storage provider and computing the MD5
    2. Stores: The underlying storage provider in which the following space resides
      1. Space containing content items: The DuraCloud space in which the content items to be considered reside
  2. Mode 2 - Generate integrity information for an item list
    1. Get integrity information from...
      1. The storage provider: Determine the file MD5 by asking the storage provider for its stored MD5 value
      2. The files themselves: Determine the file MD5 by retrieving them from the storage provider and computing the MD5
      3. Input listing name: Name of the content item which contains the listing of items over which to run the service
    2. Stores: The underlying storage provider in which the following space resides
      1. Space with input listing: The DuraCloud space in which the input listing file resides
  3. Mode 3 - Compare two integrity reports
    1. Input listing name: Name of the first content item which contains a listing of items to be compared to the second listing
    2. Second input listing name: Name of the second content item which contains a listing of items to be compared to the first listing
    3. Stores: The underlying storage provider in which the following spaces reside
      1. Space with input listing: The DuraCloud space in which the first input listing file resides
      2. Space with second input listing: The DuraCloud space in which the second input listing file resides

Service Ouputs
All outputs of this service are placed in the system space, x-service-out.

  1. bitintegrity/fingerprints-<spaceId>-<date>.csv
    • Listing of hashes when running in from space or from list modes
  2. bitintegrity/fixity-report-<listingId-0>-vs-<listingId-1>-<date>.csv
    • Comparison report of two hash listings

CloudSync

Description:

The CloudSync service starts and runs the CloudSync application, which provides capabilities to allow the backup and restore of content from a Fedora repository into DuraCloud. For more information about CloudSync, please refer to its the CloudSync documentation.

  • No labels

All content on the LYRASIS Wiki is licensed under the CC BY (Attribution) license, unless otherwise noted.