View Source

Why DuraCloud?

In the digital era, ensuring that critically important documents remain safe and available is a continual challenge. Physical computing hardware that is used to create and store documents can fail or become obsolete very quickly, providing a need for tools to ensure that these documents remain available.

There are many options for file storage and backup, with a growing trend toward the use of service providers offering off-site storage and backup solutions. These solutions are enticing, but several concerns often remain:

How do I upload files to the storage service in a simple and reliable way?
How do I ensure that files remain intact over time after I have transferred them to the storage service?
How do I ensure that the storage service that I am using receives a copy of my local files?
How do I retrieve my content once it is stored?
How do I recover a file if it has been overwritten or corrupted?
How do I make my content publicly accessible at a stable URL?
How am I protected against the storage service becoming obsolete or going away?

DuraCloud can help to solve these problems.

Amazon S3

DuraCloud is designed using Amazon S3, a robust and internationally distributed file storage service. You can think of DuraCloud as being in large part an extension of S3 that handles these concerns:

Configuring more “complex” aspects of S3 to support storage and preservation goals.
Providing additional value-added features via a set of scheduled tasks.

Preservation services are intended for long-term storage and future access, and while the technology for many of these services has been impressive, the sustainability of the services themselves has been less so. Building DuraCloud as a form of S3 proxy service backed by Amazon’s global infrastructure, we believe, provides one of the best guarantees for long-term sustainability and access to the files you are storing.

Ultimately, the goal of DuraCloud is to make the use of Amazon S3 easy for users and robust for preservation.

Configuration

Principally, these bucket configurations are automatically applied as buckets are created:

Access Control

Users can be standard or power users.

Standard users can list and upload files but cannot download or delete them.
Power users can do all of the above.

Versioning

Versioning is enabled. This supports file restore for up to 2 days post update (via request).
Learn more

Lifecycle transitions

Files are uploaded to the standard storage tier and transition to the Glacier Instant Retrieval tier after 7 days.
Learn more

Inventory

A file manifest is generated for each user-created bucket.
Learn more

Audit

A request log is generated for each user-created bucket.
Learn more

Replication

Public buckets: files in STANDARD and replicated to GLACIER (DA or Deep Archive)
Private buckets: files in GLACIER (IR or Instant Retrieval) and replicated to GLACIER (DA)
Learn more

Public access

Buckets can be created as publicly accessible. Files will be available using a URL. Files will be stored in the standard storage tier and not transitioned to Glacier; however, replication will still occur, and the backup copies will be stored in Glacier.
Learn more

Selectable region

Files can be stored in any AWS region supported by the infrastructure team or service provider.
Learn more

Note: Any requests to restore deleted or corrupted items must be made to whoever manages the DuraCloud deployment on behalf of users. An end user cannot perform these actions directly.

Scheduled tasks

In addition, these operations are run on a schedule:

Fixity checks

Performs checksum verification for each file every 6 months.

Checksum export

Checksum metadata for all files is exported every month.

File manifest

Generated on a nightly basis via the inventory configuration.

Audit log

Generated and delivered within hours of requests via logging configuration.

Storage report

Generated on a weekly basis.

Note: Generated files are retained for 30 days.