The Bridge Intake Service provides data pulled down from the Duracloud Bridge App a path into Chronopolis. It currently handles bagging of snapshots, which includes validating files from the manifest given by Duracloud.
Installation
Prereqs
- Running Duracloud Bridge App to connect to
- Staging area for creating bags
Install
- Get the latest rpm from http://adaptci01.umiacs.umd.edu/resource/bridge-intake/master/
- yum install
Installed files are as follows:
- /usr/lib/chronopolis/bridge-intake.jar
- /etc/chronopolis/application.yml
- /etc/init.d/bridge-intake
When running, the service will check for the following directories and create/apply permissions if they do not match:
- /var/log/chronopolis/
- logging data
- /var/lib/chronopolis/data/
- journaled data for tokenization
Configuration
Depending on where the data is bound, there are several configuration options available
Chronopolis
Prefix
Specifying Replicating Nodes
DPN
Sample application.yml with all properties
Release Notes
Release 2.3.2
09 August, 2018
- Fixed bug which was preventing deserialization of certain classes
- Temporarily disabled HttpFilter because of bug in Ingest Server
Release 2.3.1
07 August, 2018
- Updated logic for appending a prefix to a depositor so that it is used in every operation
Release 2.3.0
10 July, 2018
- Code licensed under BSD-3
- Use separate thread pools when executing Ingestion tasks
- Long IO operations - bagging, cleaning, etc
- Short IO operations - HTTP requests
- Bring tokenization process up to date with Chronopolis Core code
- Test that a depositor exists before attempting to push a Bag to Chronopolis or DPN
Release 2.2.1
18 May, 2018
- Fix bug where constraint for bag size was checked incorrectly
- Fix bug where tokens were not being created because a dependency was missing
Release 2.2.0
14 May, 2018
- Fix bug where we could potentially create the wrong number of replications
- Add constraint satisfaction when creating DPN Replications
- Refactor code for Chronopolis and DPN Ingest to better facilitate working on snapshots concurrently
- Break up DPN Ingestion steps into multiple classes
- Use standard lib interfaces instead of spring-batch for tasks
Release 2.1.0
8 February, 2018
- Actively clean storage upon completion of data replicating
- For chronopolis: Ingest reports PRESERVED
- For DPN: All replications completed
- Mark staging as inactive in Ingest for completed replications in Chronopolis
Release 2.0.3
29 November, 2017
- Query on Bags we created when running Tokenization
Release 2.0.2
9 November, 2017
- Remove creation of /var/log/chronopolis from rpm and instead create it on application startup
Release 2.0.1
8 November, 2017
- Use correct properties when creating Ingest Requests to Chronopolis
Release 2.0.0
30 October, 2017
- Integrate Chronopolis 2.0.0 Changes
- Import tokenization module in order to create ACE Tokens for incoming data
- Use new properties classes for configuration of storage and api communication
- Capture basic metrics when creating bags
- Update ingest flow to create bag and fixity
- Updates to build and init scripts
- Support EL6 and EL7
- Quality of life changes to the sysv init script based on rpmlint
Release 1.5.1
30 June, 2017
- Fix bug that caused replications to be created for each node in DPN
Release 1.5.0
26 June, 2017
- Add a prefix for chronopolis bound collections
- When creating replications for dpn, choose replicating nodes in a deterministic manner per snapshot so that all bags in a snapshot end up at the same DPN nodes.