Release 4.1.0
Released: July 18, 2016
The DuraCloud 4.1.0 release makes it possible to support storage in both the DPN and the Chronopolis systems independently. Support for storing content in DPN through Chronopolis remains, but this release opens the door for storing content in Chronopolis alone, for scenarios where storage in DPN is not appropriate.
The 4.1.0 release also adds several updates to better support deployment and operations, including CORS support and integration with SumoLogic logging management.
For details on specific changes which are included in the 4.1.0 release, see the JIRA issue tracker.
Release 4.0.0
Released: July 1, 2016
The DuraCloud 4.0.0 release focuses primarily on updates which allow for greater scaling and efficiency. DuraCloud will now be deployed as clustered software. While all storage accounts remain distinct, compute capacity will be shared and will scale to meet demands. This will provide increased upload and download throughput and simplify system administration, allowing for more frequent software releases.
The following changes are part of the DuraCloud 4.0.0 release:
- Both the DuraStore and DurAdmin applications can now handle requests from any DuraCloud account through a direct connection with the account database
- DuraCloud applications no longer require initialization
- DuraCloud applications can now be clustered and served behind a load balancer to support horizontal scaling for greater request throughput
- Storage reporting activities are now shared between the DuraCloud Mill and DuraStore
- The Mill manages the capture of report data and DuraStore provides a brand new set of API calls to retrieve report data
- The new DuraStore reporting API provides more flexibility in how storage reporting data can be retrieved
- Storage data will be captured daily rather than weekly
- The DuraBoss application has been retired
- The UI of the SyncTool now supports setting chunk size, to allow files up to 5 GB to be transferred without chunking
- DuraCloud applications now all require Java 8
- Upgrades to security, http, and many other project dependencies, including requirements for version consistency
- To see the full list of changes, or for more details on specific changes in release 4.0.0, see the JIRA issue tracker
Release 3.7.0
Released: January 29, 2016
The DuraCloud 3.7.0 release added the following features and updates:
User experience improvements when snapshots are taken, providing more immediate information and snapshot detail.
Improved performance for downloading manifests of large spaces. Please note: manifests are no longer sorted by default.
Improved Sync Tool performance through checkpoint frequency reduction.
- Reduced Sync Tool download size and startup time.
- Users with USER role can now view streams on open spaces.
- Improved error handling in queue deposit processing
- Improved bridge notifications
To see the full list of changes, or for more details on specific changes in release 3.7.0, see the JIRA issue tracker.
Tip | ||
---|---|---|
| ||
DuraCloud currently supports all of the major web browsers: Chrome, Safari, Firefox, and Microsoft Edge. As of version 3.7.0, DuraCloud no longer supports Microsoft Internet Explorer. This change is in response to Microsoft's recent end of support announcement for Internet Explorer. This shift allows development effort to be focused on delivering the highest quality user experience for the vast majority of our users. If you are still using Internet Explorer, we recommend you consider moving to one of the supported browsers mentioned above. |
Release 3.6.0
Released: November 23, 2015
The DuraCloud 3.6.0 release added the following features and updates:
Administrators now see a "Request Restore" button on completed snapshots. Clicking this button sends DuraCloud staff an email and logs the event in the snapshot history.
All snapshots now have a "Preservation Network Member ID" field in the snapshot details, which captures the DPN Member ID.
When listing snapshots through the bridge REST API there are now several optional parameters to limit results.
Restore actions (requesting, initiating, completing, expiring) are automatically included in the snapshot history.
Non-administrative users can now view snapshots based on the permissions assigned to the associated space. So if a user has access to a space, they also have access to see the snapshots taken of that space.
- Performing multiple snapshots at once is now supported.
To see the full list of changes, or for more details on specific changes in release 3.6.0, see the JIRA issue tracker.
Release 3.5.0
Released: October 19, 2015
The DuraCloud 3.5.0 release focuses on smoothing the integration with DPN. The key features that were added that are visible in DuraCloud:
- Snapshot details, history, and artifacts are downloadable.
- Manifest and audit logs are now visible to anyone with access to a space.
- "Single Pass" mode for users running the DuraCloudSync tool in the GUI mode.
- A snapshot action can now be restarted or cancelled, and any errors that occur while performing a snapshot can be reported properly.
- Improved handling of chunked large files in snapshots.
To see the full list of changes and bug fixes, or for more details about specific changes in release 3.5.0, see the DuraCloud Sprint and DuraCloud Vault Sprint
Release 3.4.0
Released: August 14, 2015
The DuraCloud 3.4.0 release focuses on smoothing the integration with DPN. The key features that were added that are visible in DuraCloud:
Restore function is limited to root users.
Any users with read access can request a restore.
- Metadata, added to the snapshot on completion, is viewable in the via the API as well as DuraAdmin.
- Jump start feature in the synctool increases upload throughput by not checking for the existence of an item in DuraCloud before uploading it.
- Security was added to the bridge api
- Content in restored spaces expire after a period of time and are automatically cleaned up.
To see the full list of changes, or for more details on specific changes in release 3.4.0, see the JIRA issue tracker.
Release 3.3.0
Released: June 15, 2015
The DuraCloud 3.3.0 release adds two primary features:
- Secure media streaming: There is now the option to stream audio and video files from a DuraCloud space using either open or secure streaming. Open streaming provides consistent URLs for streamed content items, which is ideal for open access data, as anyone with the URL can stream the content. Secure streaming, in contrast, allows a signed URL to be generated each time a file is to be streamed. These signed URLs can be used for only a certain amount of time, and can also be limited to a particular IP range. Secure streaming allows for streaming of media content to a limited audience.
- Two factor authentication: DuraCloud will now accept an IP address, IP range, or list of IP ranges from which requests must originate for a given user to successfully log in to their DuraCloud account. This adds a second layer of security by disallowing any attempts to log in to DuraCloud which do not come from the expected internet address. This can be used, for example, to ensure that attempts to log in to a DuraCloud account always occur from an on-campus location.
Beyond the two primary security-focused features, many updates were made to further support the DuraCloud Mill and to resolve a variety of bugs.
To see the full list of changes, or for more details about specific changes in release 3.3.0, see the JIRA issue tracker.
Release 3.2.0
Released: December 17, 2014
The DuraCloud 3.2.0 release is focused on two main themes: Integration with DPN, and integration with the DuraCloud Mill.
- DPN Integration: A new Snapshot Storage Provider was added to DuraCloud, which is used to integrate with Chronopolis, allowing DuraCloud and Chronopolis to function together as a single DPN node.
- DuraCloud Mill Integration: The back end of DuraCloud, where all of the bits are processed, has moved into system called the DuraCloud Mill. This system manages the audit processing, manifest generation, duplication, and bit integrity functions of the DuraCloud hosted service. This version of DuraCloud transitioned many of those functions to the Mill, and added REST API and UI components which allow users to interact with the Mill to retrieve things like space audit logs and manifest files.
Beyond the two major themes, this release includes significant housekeeping work as well. The full set of project dependencies were reviewed, updated, and significantly pruned. Each piece of the DuraCloud software was reviewed to ensure its configuration and dependency set were up-to-date and correct. Much of this was in preparation for the DuraCloud software being hosted by Sonatype, which means that the 3.2.0 release is the first DuraCloud release to be available in Maven Central. This makes the DuraCloud software much easier to reuse in other Java projects.
Also notable is a new "jump start" feature in the DuraCloud SyncTool which enables users to dramatically increase their throughput for first time uploads of large numbers of small files. See DuraCloud Sync Tool or DuraCloud Sync Tool - Command Line for details.
To see the list of housekeeping tasks and bug fixes in release 3.2.0, see the JIRA issue tracker. The work completed on the DuraCloud Mill is maintained in a stand-alone JIRA project. The work on the Chronopolis integration is also in a stand-alone JIRA project.
Release 3.1.0
Released: June 6, 2014
The primary features of release 3.1.0 are:
- The DuraCloud SyncOptimize Tool, a new tool that can be accessed both as a command-line tool as well as through the Sync Tool User interface, helps you optimize throughput from your local machine when uploading content to DuraCloud.
- The Sync Tool UI now enables users to manually set the thread count to improve throughput performance.
- The Sync Tool now supports the use of a prefix. If a prefix is used, that value is added to the beginning of all content IDs created in DuraCloud by the Sync Tool. For details on how it works, see DuraCloud Sync Tool orDuraCloud Sync Tool - Command Line.
To see the full list of changes, or for more details about specific changes in release 3.1.0, see the JIRA issue tracker.
Release 3.0.0
Released: April 22, 2014
The DuraCloud 3.0.0 release is the first step towards allowing DuraCloud to be run in a more distributed and efficient manner. Towards that end, the services framework has been removed in favor of handling the same tasks (primarily duplication between providers and bit integrity checking) through a scalable processing engine which is run outside of the DuraCloud instances themselves. This provides for better performance on the instances and better performance for the services, as well as less effort for DuraCloud users.
The primary features of release 3.0.0 are:
- Users of DuraCloud no longer need to manage services. The service framework has been replaced by a back-end task processing system that leverages the massively parallel processing capabilities of the cloud.
- Library updates, to keep the primary components of DuraCloud software up-to-date. Examples: Upgraded from Maven 2 build system to Maven 3. Upgraded the Spring Framework version from 2.5 to 4.0.2.
- The auditing framework's use of ActiveMQ have been removed in favor of a more pluggable architecture using an external queuing system. Audit event logging is now being handled by the same task processing framework that is managing duplication and bit integrity checks.
- Media streaming capabilities updated to continue to work just as before, but without the need for the Media Streamer service.
To see the full list of changes, or for more details about specific changes in release 3.0.0, see the JIRA issue tracker.
Release 2.4.0
Released: September 20, 2013
...
.4.0
Released: September 20, 2013
The primary features of release 2.4.0 are:
- New drag and drop capability for file upload
- DuraCloud now provides the option to upload files using the familiar file selector or with simple drag and drop.
- This new functionality is enabled using HTML5, which removes the need for a java browser plugin.
- DuraCloud now provides the option to upload files using the familiar file selector or with simple drag and drop.
- New installers for the Sync Tool
- The Sync Tool can now be installed using a platform-specific installer on Windows, Mac, and Linux.
- The installers provide a simple graphical installation flow and are consistent with the usual installation process on each operating system.
- The installers provide the option to start up the Sync Tool on system restart, to ensure that the Sync Tool will continue to run even if you need to reboot your computer.
- The installers add links and shortcuts which make it easy to find the Sync Tool when you need to see progress or change settings.
- New features for the Sync Tool
- The Sync Tool now provides 3 options for handling files which have changed locally:
- Overwrite the file (the default, and the only option available in previous versions)
- Skip the file (ensures that there are no changes made to any existing files in DuraCloud)
- Rename original (rename the old copy of this file first, then add the new file to DuraCloud)
- The Sync Tool now allows the DuraCloud password to be provided via a system property or via a prompt, both of which remove the need to include a password as part of the command line call
- The Sync Tool now provides 3 options for handling files which have changed locally:
- New features for the Retrieval Tool
- The Retrieval Tool can now retrieve a list of the files in a space
- The Retrieval Tool can now retrieve a specific set of files from a space, as well as being able to retrieval all files in a space
- The Retrieval Tool now allows the DuraCloud password to be provided via a system property or via a prompt, both of which remove the need to include a password as part of the command line call
To see the full list of changes, or for more details about specific changes in release 2.4.0, see the JIRA issue tracker.
Release 2.3.1
Released: March 28, 2013
...
The primary features of release 1.2 are:
- Upload Tool
- Provides a graphical method for transferring large numbers of files to DuraCloud.
- Allows for the selection of both files and folders for transfer, and presents a visual indication of the transfer progress.
- Can be run from the DuraCloud UI by choosing the "Add Many Items" button when a space is selected.
- Can also be download and run locally.
- SDSC Storage Provider
- An initial beta release of the new storage provider integration for connecting to the SDSC Cloud storage system.
...
- The Duplicate on Ingest service is now the Duplicate on Change service.
- This service now supports all of the same on-ingest features as before, but it now also performs duplication of all update and delete actions as well. This allows the primary and secondary cloud stores to be kept completely in sync.
- The Bulk Bit Integrity Service has been improved.
- This service has been updated and verified to properly handle spaces with up to 1 million items
- The second step of the MD5 verification, which used to run locally on the instance, has been moved to hadoop, allowing the service to complete much more quickly for large data sets.
- User management functions have been removed, as they are now performed by the DuraCloud Management Console.
- As a convenience, administrators are still able to see the list of users and their roles within the DuraCloud Administrator UI.
- Service outputs have been made more consistent.
- All DuraCloud services which produce an output file now store that file in the x-service-out space.
- Services which produce log files store those logs in the x-service-work space.
- The names of the output files have been made more consistent, making it simpler to determine which files correspond to which service deployment.
- Password security has been improved.
- All passwords used within DuraCloud are now immediately pushed through a hashing function before being are stored, so that no user passwords are transferred or stored as clear text.
- A ServiceClient is now available, to compliment the StoreClient and make it easier to make direct API calls to manipulate DuraCloud services.
...
- A new Retrieval Tool, a companion to the existing Sync Tool, which is a command-line tool for the retrieving content from DuraCloud spaces.
- A new Bulk Bit Integrity Checker service, which can be run over content stored in Amazon to create a listing of checksums calculated for each file. This new service pairs well with the Bit Integrity Checker service (previously known as the Fixity Service), allowing the heavy processing to be handled in parallel using Hadoop on an Amazon EC2 cluster, and the simpler checks and comparisons to be handled by the DuraCloud instance.
- A new Duplicate on Demand service, which can be used to copy files from the primary Amazon store into another storage provider. This service pairs well with the Duplicate on Upload service (previously known as the Replication Service) by performing the large up-front copy using Hadoop on an Amazon EC2 cluster, then allowing Duplicate on Upload to watch for and add new files as they are uploaded.
- Integration of a new storage provider: Microsoft Windows Azure.
...