Page History

...

...

...

...

The Replication Task Suite currently supports the following versions of DSpace software:

Installation instructions for each version are included below:

• 81953514
• Installation ReplicationTaskSuite#Installation on DSpace 65.x
• Installation ReplicationTaskSuite#Installation on DSpace 3.x , or 4.x or 5.xInstallation on DSpace 1.8.x
• 81953514

User Interface Compatibility Notes

...

1. In your DSpace Source directory ([dspace-src]), you will need to modify the following POM file:
   

   • [dspace-src]/dspace/modules/additions/pom.xml (This POM will ensure that the "dspace-replicate" dependency is made available to commandline and ALL DSpace interfaces)

2. For this pom.xml file, add the following <dependency> section at the end of the existing <dependencies> section (just before the closing </dependencies> tag). NOTE: the exclusions are required to work around DS-3536.

   Code Block

   <dependencies> ... <!-- Adding this dependency will install the Replication Task Suite Addon --> <dependency> <groupId>org.dspace</groupId> <artifactId>dspace-replicate</artifactId> <version>6.0<1</version> <!-- These exclusions are currently necessary to resolve dependency mismatches with some dependencies pulled into RTS 6.0 to work with DuraCloud, see DS-3536 for details --> <exclusions> <exclusion> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> </exclusion> <exclusion> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-core</artifactId> </exclusion> <exclusion> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpmime</artifactId> </exclusion> <exclusion> <groupId>org.springframework</groupId> <artifactId>spring-expression</artifactId> </exclusion> <exclusion> <groupId>org.springframework.security</groupId> <artifactId>spring-security-core</artifactId> </exclusion> <exclusion> <groupId>org.codehaus.jackson</groupId> <artifactId>jackson-mapper-asl</artifactId> </exclusion> <exclusion> <groupId>org.codehaus.jackson</groupId> <artifactId>jackson-core-asl</artifactId> </exclusion> </exclusions> </dependency> </dependencies>

   
   

3. Once you've finished modifying both pom.xml files, rebuild DSpace by running the following from your [dspace-src]/dspace/ folder:

   Code Block
   mvn clean package

   
   

4. Update the default dspace.cfg to include the Replication Task Suite config files. This ensures these configs are loaded as part of your DSpace configuration. This also allows you to override the configurations in your own local.cfg file.

   Code Block
   include = ${module_dir}/replicate.cfg include = ${module_dir}/replicate-mets.cfg

   1. You should ensure these configurations exist in your [dspace-src]/dspace/config/ directory.  That way they will be auto-installed/copied whenever you run "ant update" (see next step).
5. Follow the instructions in the Configuration81953514 section below in order to enable & configure the Replication Task Suite Add-On.

6. You will need to update your existing DSpace 3.x installation, by running the following from your [dspace-src]/dspace/target/dspace-[version]-build/ directory

   Code Block
   ant update

   
   

   Note
   Alternatively, if you don't want to do a full DSpace update, you can just update your existing binaries & webapps by running the following two commands: ant update_code (Updates the existing [dspace]/lib/ directory) ant update_webapps (Updates the existing [dspace]/webapp/ directory)

   
   

Installation on DSpace

...

5.x

• Follow the instructions for deployment on DSpace 6.x

...

• above, substituting version 5.0 of the dspace-replicate dependency.

Installation on DSpace 3.x or

...

4.x

1. In your DSpace Source directory ([dspace-src]), you will need to modify the following POM file:
   

   • [dspace-src]/dspace/modules/additions/pom.xml (This POM will ensure that the "dspace-replicate" dependency is made available to commandline and ALL DSpace interfaces)

2. For this pom.xml file, add the following <dependency> section at the end of the existing <dependencies> section (just before the closing </dependencies> tag).

   Code Block
   <dependencies> ... <!-- Adding this dependency will install the Replication Task Suite Addon --> <dependency> <groupId>org.dspace</groupId> <artifactId>dspace-replicate</artifactId> <version>3.4</version> </dependency> </dependencies>

   
   

3. Once you've finished modifying both pom.xml files, rebuild DSpace by running the following from your [dspace-src]/dspace/ folder:

   Code Block
   mvn clean package

   
   

4. Follow the instructions in the Configuration 81953514 section below in order to enable & configure the Replication Task Suite Add-On.
   
   1. You may wish to ensure these configurations exist in your [dspace-src]/dspace/config/ directory.  That way they will be auto-installed/copied whenever you run "ant update" (see next step).

5. You will need to update your existing DSpace 3.x installation, by running the following from your [dspace-src]/dspace/target/dspace-[version]-build/ directory

   Code Block
   ant update

   
   

   Note
   Alternatively, if you don't want to do a full DSpace update, you can just update your existing binaries & webapps by running the following two commands: ant update_code (Updates the existing [dspace]/lib/ directory) ant update_webapps (Updates the existing [dspace]/webapp/ directory)

   
   

...

...

1. In your DSpace Source directory ([dspace-src]), you will modify two Maven pom.xml files:
   
   • [dspace-src]/dspace/pom.xml (This POM controls dependencies of CommandLine scripts. Modifying it will let you run dspace-replicate from commandline)

   • [dspace-src]/dspace/modules/xmlui/pom.xml (This POM controls dependencies of XMLUI. Modifying it will let you run dspace-replicate from XMLUI)

2. For each of these pom.xml files, add the following <dependency> section at the end of the existing <dependencies> section (just before the closing </dependencies> tag).

   Code Block
   <dependencies> ... <!-- Adding this dependency will install the Replication Task Suite Addon --> <dependency> <groupId>org.dspace</groupId> <artifactId>dspace-replicate</artifactId> <version>1.3</version> </dependency> </dependencies>

   
   

3. Once you've finished modifying both pom.xml files, rebuild DSpace by running the following from your [dspace-src]/dspace/ folder:

   Code Block
   mvn clean package

   
   

4. Follow the instructions in the Configuration 81953514 section below in order to enable & configure the Replication Task Suite Add-On.
   
   1. You may wish to ensure these configurations exist in your [dspace-src]/dspace/config/ directory.  That way they will be auto-installed/copied whenever you run "ant update" (see next step).

5. You will need to update your existing DSpace 1.8.x installation, by running the following from your [dspace-src]/dspace/target/dspace-[version]-build/ directory

   Code Block
   ant update

   
   

   Note
   Alternatively, if you don't want to do a full DSpace update, you can just update your existing binaries & webapps by running the following two commands: ant update_code (Updates the existing [dspace]/lib/ directory) ant update_webapps (Updates the existing [dspace]/webapp/ directory)

   
   

...

Follow the latest installation instructions, based on the version of DSpace you are running:, based on the version of DSpace you are running:

• 81953514
• 81953514
• 81953514
• Installation on DSpace 6.x
• Installation on DSpace 3.x, 4.x or 5.x
• Installation on DSpace 1.8.x

Once you have reinstalled the Replication Task Suite, you should compare your existing configurations with the latest Replication Task Suite configurations.  In most cases, your existing configurations should function perfectly, but you should review the differences just in case.

...

1. A copy of all configuration files utilized by the Replication Task Suite (RTS) can be found in the following locations:
   
   1. Configs for RTS version 1.x : https://github.com/DSpace/dspace-replicate/tree/dspace-replicate-1_x/config/modules
   2. Configs for RTS version 3.x : https://github.com/DSpace/dspace-replicate/tree/dspace-replicate-3_x/config/modules
   3. Configs for RTS version 6.x : https://github.com/DSpace/dspace-replicate/tree/master/config/modules
2. Copy the following configuration files to your DSpace's [dspace]/config/modules/ directory:
   
   1. replicate.cfg - This file contains the base settings for the Replication Task Suite
   2. replicate-mets.cfg - This file provides a few additional replication options specific to METS-based AIPs (see below for more details)
   3. duracloud.cfg - If you'd like to replicate/backup your content to DuraCloud, this file holds your DuraCloud account information
3. Edit your [dspace]/config/modules/curate.cfg configuration file to define & enable all tasks. The list of tasks to add to this configuration file depends on which type of AIP (METS based or BagIt based) you wish to use. Please see the AIP Format Options 81953514 section below for the details of what should be added to your curate.cfg file
   
   1. A sample, fully enabled curate.cfg configuration file is provided alongside the other Replication Task Suite config files listed above.  This sample file is preconfigured to use METS-based AIPs.
4. Recommended (but not required):  Edit your [dspace]/config/modules/dspace.cfg and enable the Replication Task Suite 'listener' to perform automatic synchronization of your AIP backup store with what is in DSpace (see Automation Options for more info).

Overview of Configuration Options

Before getting started, you may wish to determine the answers to the following questions:

1. AIP Format Options81953514: Does you institution want to backup using the default DSpace AIP format (METS packaging)? Or would you rather utilize the new BagIt AIP Format?
2. Storage Options81953514: Does you institution plan to use the Replication Suite to backup to a local/mounted drive? Or would you like to connect it to a DuraCloud account?
3. Automation Options (Recommended)81953514: Do you want to automatically sync your AIP backup store with what is in DSpace? (this is highly recommended, but not required)
4. Additional Options81953514: Do you plan to use Checkm manifests for checksum auditing?

...

One of the first questions to ask yourself is the format you wish to utilize for your AIPs.

There are two options:

1. DSpace AIP Format (METS-based) (default) - This is the same AIP format utilized by the DSpace AIP Backup and Restore feature, so it is 100% compatible with that DSpace feature. In fact when using this format, the Replication Task Suite just "wraps" calls to the AIP Backup and Restore feature itself.
2. BagIt AIP Format (beta) - This is a new AIP format provided by the Replication Task Suite. It generates AIPs in the BagIt File Packaging Format. Institutions which already are familiar with BagIt or use it elsewhere may find this format preferable.  (Please note that this AIP format does not yet support all DSpace objects. See the below table for more information.)

These two AIP formats are not identical.  The below table seeks to describe some of the differences.

For more information on the tasks available based on your AIP format choice, please see the Problem Statement and Usage Examples 81953514 section below. This section also provides good examples of how to use each of the tasks available to you in the Replication Task Suite.

...

Where your AIPs will be stored is the next decision to make. There are three options currently available:

1. Local Storage: Replicate/Backup content to another location (folder) on your local filesystem.
2. Mountable Storage: Replicate/Backup content to a mounted external filesystem (e.g. NFS-mounted drive).
3. DuraCloud Storage: Replicate/Backup content to an existing DuraCloud account.

...

Before configuring a local storage option, please ensure you have enough space available on your local hard drive (or mounted drive/SAN if your local folder is actually remote storage). You can use the "Estimate Storage Space for AIP(s)" (estaipsize) task to estimate the amount of new storage space you will need.

...

• Before you can use the DuraCloud Storage plugin, you first must signup for a DuraCloud account (or signup for a trial account). 
• Once you have a DuraCloud account, you can configure the Replication Task Suite to use your DuraCloud Account Settings 81953514 (as detailed below).
• In DuraCloud, you will also want to create one (or more) "DuraCloud Spaces" in which to store your DSpace AIPs. You'll then need to configure those space(s) in the DuraCloud Storage Settings 81953514 of the Replication Task Suite (as detailed below).  The DuraCloud Space represents the location in your DuraCloud account where you want to DSpace to store its content.  Having a separate DuraCloud Space for your DSpace content is recommended (though not required), as it allows you to separate your DSpace content from any other content you may wish to store in DuraCloud.

...

1. For each DSpace object (Community, Collection, Item), an AIP zip file is generated on the server running DSpace.  The AIP is temporarily stored in the server's [dspace]/replicate/[group.aip.name] directory, where "[group.aip.name]" is the value of the "group.aip.name" setting in your "replicate.cfg" configuration file (see DuraCloud Storage Settings 81953514 below for more info).  This "group.aip.name" setting also corresponds to the ID of the DuraCloud Space where the AIP will be stored.
2. Once the AIP is generated, the Replication Task Suite determines whether a file of this same name already exists in the DuraCloud Space.
   
   1. If this file does not exist in DuraCloud, the locally generated AIP is uploaded to DuraCloud.
   2. If a file of this name already exists, then the Replication Task Suite checks to see if it differs from the locally generated AIP. It does so by verifying the DuraCloud reported checksum with the locally generated checksum.
      
      1. If the AIP checksums differ, the locally generated AIP is uploaded to DuraCloud and it replaces the version that was previously in DuraCloud.
      2. If the AIP checksums are identical, then the AIP is skipped. Nothing is uploaded to DuraCloud as the files are identical. This ensures that unnecessary uploads to DuraCloud are avoided.
3. Once the local copy of the AIP is no longer needed, it is removed from the server's temporary location.
4. If an upload to DuraCloud occurred, the local "odometer" is incremented to ensure it always details the total amount of content that has been uploaded (see Keeping Score 81953514 section for more info on the "odometer").

...

1. For each DSpace object (Community, Collection, Item), that object's AIP is downloaded from DuraCloud to the server running DSpace (the appropriate AIP is located in DuraCloud via its filename).  The AIP is temporarily stored in the server's [dspace]/replicate/[group.aip.name] directory, where "[group.aip.name]" is the value of the "group.aip.name" setting in your "replicate.cfg" configuration file (see DuraCloud Storage Settings 81953514 below for more info).  This "group.aip.name" setting also corresponds to the ID of the DuraCloud Space where the AIP is stored.
2. Once the download completes, the local "odometer" is incremented to ensure it always details the total amount of content that has been downloaded (see Keeping Score 81953514 section for more info on the "odometer").
3. The AIP is then "unzipped", and the DSpace object is restored/replaced as needed.
4. Once the local copy of the AIP is no longer needed, it is removed from the server's temporary location.

...

1. Enable DuraCloud Storage Plugin: Ensure the Replication suite is setup to use the 'DuraCloudObjectStore' plugin

   Code Block
   # Replica store implementation class (specify one) plugin.single.org.dspace.ctask.replicate.ObjectStore = \ org.dspace.ctask.replicate.store.DuraCloudObjectStore

   
   

2. Configure DuraCloud Primary Space to use: Your DuraCloud account allows you to separate content into various "Spaces". You'll need to create a new DuraCloud Space that your AIPs will be stored within, and configure that as your group.aip.name (by default it's set to a DuraCloud Space with ID of "aip-store").

   Code Block
   # The primary storage group / folder where AIPs are stored/retrieved when AIP based tasks # are executed (e.g. "Transmit AIP", "Restore from AIP") group.aip.name = aip-store

   
   

3. Optionally, Configure Additional DuraCloud Spaces: If you have chosen to utilize Checkm manifest validation, you will need to create and configure a DuraCloud Space corresponding to the group.manifest.name setting below. Additionally, if you have chosen to enable the Automatic Replication, you will need to create and configure a DuraCloud Space corresponding to the group.delete.name setting below.

   Code Block

   # The storage group / folder where Checkm Manifests are stored/retrieved when Checkm Manifest # based tasks are executed (org.dspace.ctask.replicate.checkm.*). group.manifest.name = manifest-store # The storage group / folder where deletion records are kept when an object deletion occurs # and the ReplicationConsumer is enabled (see below). Each time an object is deleted in DSpace, # a DELETION-RECORD@[handle] file is written to this location. The deletion record is always in # BagIt format. It details basic info about the deleted object (along with any deleted child/member objects) # This deletion record may be used to restore those deleted object(s) at a later time (using "Restore from AIP" tasks), # or may be used to permanently remove their AIP(s) from storage (using "Remove AIP" task). group.delete.name = deletions

   
   

   Info
   title Using File Prefixes instead of separate DuraCloud Spaces
   If you'd rather keep all your DSpace files in a single DuraCloud Space, you can tweak your "group.aip.name", "group.manifest.name" and "group.delete.name" settings to specify a file-prefix to use.  For example: group.aip.name = dspace-backup/aip-store group.manifest.name = dspace-backup/manifest-store group.delete.name = dspace-backup/deletions With the above settings in place, all your DSpace content will be stored in the "dspace-backup" Space within DuraCloud.  AIPs will all be stored with a file-prefix of "aip-store/" (e.g. "aip-store/ITEM@123456789-2.zip").  Manifests will all be stored with a file-prefix of "manifest-store/".  And any object deletion records will be stored with a file-prefix of "deletions/".   This allows you to keep all your content in a single DuraCloud Space while avoiding name conflicts between AIPs, Manifests and deletion records.

   
   

...

The Replication Task Suite offers several options to automate replication of content to your backup storage location of choice.

1. Automatically Sync Changes (via Queue) 81953514 : Any changes that happen in DSpace (new objects, changed objects, deleted objects) are automatically added to a "queue". This queue can then be processed on a schedule (via cron).
2. Scheduled Site Auditing/Replication 81953514 : You may also wish to perform a full site audit or backup on a scheduled basis.

...

• METS-based AIP Replicate Consumer: This consumer will listen for changes to any DSpace Communities, Collections, Items, Groups, or EPeople.  It should be utilized if you have chosen to use METS-based AIPs. See AIP Format Options 81953514 above for more details.

  Code Block

  #### Event System Configuration #### # ADD the "replicate" consumer to the end of the list of 'default.consumers' (This enables the consumer) event.dispatcher.default.consumers = versioning, search, browse, discovery, eperson, harvester, replicate .... # Configure consumer to manage METS AIP content replication event.consumer.replicate.class = org.dspace.ctask.replicate.METSReplicateConsumer event.consumer.replicate.filters = Community|Collection|Item|Group|EPerson+All

   

  
  

  • In human terms, this configuration essentially means: listen for all changes to Communities, Collections, Items, Groups and EPeople. If a change is detected, run the "METSReplicateConsumer" (which adds that object to the queue).

• BagIt-based AIP Consumer : This consumer will ONLY listen for changes to DSpace Communities, Collections and Items as those are the only types of objects which are stored in BagIt-based AIPs. See AIP Format Options 81953514 above for more details

  Code Block

  #### Event System Configuration #### # ADD the "replicate" consumer to the end of the list of 'default.consumers' (This enables the consumer) event.dispatcher.default.consumers = versioning, search, browse, discovery, eperson, harvester, replicate .... # Configure consumer to manage BagIt AIP content replication event.consumer.replicate.class = org.dspace.ctask.replicate.BagItReplicateConsumer event.consumer.replicate.filters = Community|Collection|Item+Install|Modify|Modify_Metadata|Delete

   

  
  

  • In human terms, this configuration essentially means: listen for any new, modified or deleted Items, Collections and Communities. If you do not care about Community or Collection AIPs, just remove 'Community' or 'Collection' from the list. When one of the specified changes is detected, run the "BagItReplicateConsumer" (which adds that object to the queue).

...

• Both "add" and "modification" events add the "transmitsingleaip" task (which will regenerate & transmit the object AIP to replica storage) to the queue of tasks to perform.  Please ensure you are scheduling this queue to be processed, as detailed in  Processing the Consumer Queue 81953514 below.
• The "delete" event triggers a special "catalog" task.  This "catalog" task does the following:
  
  • First, it creates a plaintext "catalog" file which lists all the objects that were deleted.
  • Second, it moves the AIPs for those deleted objects to the "group.delete.name" storage area (this is essentially putting them in a "trash" folder, where they can be cleaned up later, or potentially restored if the deletion was accidental).
• By default, the queue used for all replication events is located at : [dspace]/ctqueues/replication  (this is a plaintext file which just lists all actions that should be performed the next time the queue is processed)

...

Processing the Sync Consumer Queue

...

...

Enhancing the Performance of the Queue Processing (optional)

...

In DSpace, by default, duplicate tasks in a Curation System queue will each be processed individually. So, that means if an Item is updated 10 times, it will appear in the queue 10 times, and its AIP will be (re-)generated and (re-)transmitted to storage 10 times when that queue is processed.  (DuraCloud Note: Some storage platforms, e.g. DuraCloud, provide a way to determine whether a newly generated AIP actually differs from the one in replica storage. So, in the case of DuraCloud storage, the AIP will be re-generated 10 times, but it will only be transmitted to DuraCloud ONCE. The other 9 times, the DuraCloud storage plugin will determine that the checksum of the new AIP is identical to the one in DuraCloud and skip the transmission step.  See How DuraCloud storage works 81953514 section above for more info.)

...

We can suppose our data curator has identified a collection of items in her DSpace repository consisting of high-value, born-digital, and unique/irreplaceable (not held elsewhere) content (called the 'Amazing Images' collection). She prudently wishes to insure against catastrophic local loss of this content by keeping a copy or replica of this collection elsewhere (e.g. either on a backup drive, or even in the cloud via a service like DuraCloud). She'd prefer to replicate all her DSpace content, but realizes that storage costs over long periods has made her administration wary, so decides to begin with this collection.

First Steps - Estimation

In order to budget for replication storage, she needs to know the 'size' of the collection. When she asks her sysadmin, he replies that it is easy to give her figures for the whole DSpace asset store, but since collections aren't stored separately, she would have to add up each item's bitstreams in the collection, a rather tedious process. Thus the first task: a reporting tool which operates on natural DSpace objects, rather than storage volumes. The "Estimate Storage Space for AIP(s)" (estaipsize) task will give her this ability.

...

We should warn that the estimates from this task are rather crude, in that they do not measure the actual size of all AIPs. Rather they just total up the bitstream (file) sizes (and do not include metadata files). However, even this crude estimate should provide a decent idea of overall storage needs.

Replicating

Having secured approval to replicate 'Amazing Images' collection, our curator obviously needs a task to generate the AIP representations of each item in the collection, and transmit these archive files to the replication storage site (which may be service-backed, local, in the cloud, etc, as will be explored below).  This task is the "Transmit AIP(s) to Storage" (transmitaip) task.

...

Our data curator may elect to perform this task in the DSpace Admin UI, or, if the collection is rather large, she may instead 'queue' the task for later execution by using the queueing facility available in the curation system. We should note that the 'transmitaip' task, like all other replication tasks, operates on whatever DSpace object(s) they are given. Thus, if the object is a collection, the task creates (and transmits, of course) an AIP for the collection object itself (metadata and logo), as well as AIPs for each item in the collection. If the task is given an identifier for a single Item, then only one AIP will be created and transmitted.

Verifying Replication

While the 'transmitaip' task will report on whether or not it was successful in generating and transmitting AIP(s) to the replication service, our data curator wants the ability (within DSpace) to check whenever she likes that the AIP(s) which were transmitted are still there. A simple task "Verify AIP(s) exist in Storage" (verifyaip) can perform this function.

Ensuring Replica Integrity and Accuracy over time

The 'Amazing Images' collection is comparatively static, meaning that few new items are likely to be added, and most of the metadata in each item is not routinely changed. However, over longer periods of time, cataloging errors are discovered and corrected, perhaps formats become obsolete and new bitstreams are added. If the curator is fastidious about each change, and performs the 'transmitaip' task on each item that has changed, then in general the set of AIP replicas will always be 'in sync' with the repository. However, it useful to have the means to ensure that the replicas agree with the repository without having to create and transmit entirely new ones. Thus the task: "Audit against AIP(s)" (auditaip), which can also be thought of as a simple, quick auditing task. When performed on an Item, the task does the following:

...

A set of replication tasks perform these functions, as described below.

Restoring Object(s)

If the curator should ever find the need to restore a deleted object, a variety of restoration based tasks are available.  The base task is the "Restore Missing Object(s) from AIP(s)" (restorefromaip) task.

...

• Restore Single Object from AIP (restoresinglefromaip)
  
  • This task acts the same as the default "restorefromaip" task, but it does NOT restore any child objects. So, if it is run on a collection, just the collection itself will be restored (items in that collection will not be restored).
• Restore Missing Object(s) but Keep Existing Objects (restorekeepexisting)
  
  • This task acts similar to the default "restorefromaip" task, but it attempts to skip over any objects which already exist in the repository. In other words, an error is not thrown if an object already exists – rather that entire object (and all its child objects) are skipped over during processing and left unchanged. This mode is identical to the "Keep Existing" mode of the DSpace AIP Backup and Restore tool.

Replacing Object(s)

If the curator should ever find a need to replace a corrupted object or revert an existing object back to the version in remote storage, a variety of replacement tasks are available.  The base task is the "Replace Existing Object(s) with AIP(s)" (replacewithaip) task.

...

• Replace Single Object from AIP (replacesinglewithaip)
  • This task acts the same as the default "replacewithaip" task, but it does NOT replace any child objects. So, if it is run on a collection, just the collection metadata will be replaced (items existing in that collection will not be replaced).

Cleanup

Ordinarily, a replication arrangement is long standing: the preservation function cannot be fulfilled unless the replicas (here, the AIPs) are always kept and available. However, some collections (or items within them) may be removed for a variety of reasons: legal challenge, de-accession, etc. When the repository no longer locally wants to hold the object, the replica AIP ceases to have value. The task 'Remove AIP(s) from Storage' (removeaip) will permanently delete the replica store AIP for its identifier. As will other replication tasks, if the identifier points to collection or community, all the AIPs of all the members will also be permanently deleted.

Keeping Score

Many storage providers have cost structures that are more complex than simple functions of the total stored bytes: particularly cloud providers have costs associated wth the use of the network to upload and download the stored object. An object that occupies 2 megaBytes might cost far more over time than a 1 gigaByte object, if the former is downloaded 1000 times for every time the latter is. The replication system provides a very rudimentary task to help manage and track these factors: 'Read Odometer' (readodometer). This task simply displays the readings from the replication system that records cumulative use. The statistics are:

...

More information about each of these storage options (and how to configure them) is available in the Storage Options 81953514 configuration section above.

...

1. Download the Replication Suite code from GitHub: https://github.com/DSpace/dspace-replicate
   

   1. Checkout the branch you wish to develop against.  For example, to checkout the 1.x branch of the codebase:

      Code Block
      git checkout dspace-replicate-1.x

      
      

2. Build/Compile the Replication Suite, by running the following from the root directory

   Code Block
   mvn package

   
   

3. The code will be compiled into a JAR and all its dependencies will also be copied to your "target" directory
   
   1. The main dspace-replicate.jar will be compiled to:
      
      • [dspace-replicate]/target/dspace-replicate-[version].jar (The Replication Suite Plugin)
   2. There will also be a total of 4 dependency JARs that will be copied to:
      
      • [dspace-replicate]/target/lib/common-[version].jar (DuraCloud common libraries - required for DuraCloud integration)
      • [dspace-replicate]/target/lib/commons-compress-[version].jar (Apache Commons Compress - prerequisite for Replication Suite plugin)
      • [dspace-replicate]/target/lib/storageprovider-[version].jar (DuraCloud storage provider libraries - required for DuraCloud integration)
      • [dspace-replicate]/target/lib/storeclient-[version].jar (DuraCloud store client libraries - required for DuraCloud integration)
4. Once the codebase is compiled, you can install it by following the Installation 81953514 instructions above.  
   
   1. Alternatively, you may temporarily copy all 5 JARs (dspace-replicate + dependency JARs) to the following locations for testing purposes only:
      
      • DSpace "lib" folder (e.g. [dspace]/lib/) - This will make the Replication Task Suite available via the commandline
      • DSpace XMLUI "lib" folder (e.g. [dspace]/webapps/xmlui/WEB-INF/lib/) - This will make the Replication Task Suite available via the XMLUI.
   2. You will also need to follow the Configuration 81953514 instructions above in order to properly enable & configure the Replication Task Suite.