The Bridge Intake Service provides data pulled down from the Duracloud Bridge App a path into Chronopolis. It currently handles bagging of snapshots, which includes validating files from the manifest given by Duracloud.
Links
Installation
Prereqs
- Running Duracloud Bridge App to connect to
- Staging area for creating bags
Install
- Get the latest rpm from http://adaptci01.umiacs.umd.edu/resource/bridge-intake/master/
- yum install
RPM
Installation can be done through the provided rpm. This offers builds for both RHEL6 and RHEL7.
Installation Notes
Installed Files For RHEL6
/etc/init.d/bridge-intake
/usr/local/chronopolis/intake/dc
/usr/local/chronopolis/intake/dc/application.yml
/usr/local/chronopolis/intake/dc/bridge-intake.jar
Installed Files For RHEL7Installed files are as follows:
/usr/lib/
...
systemd/system/bridge-intake.
...
service /usr/local/chronopolis/intake/dc /usr/local/chronopolis/intake/dc/application.yml
...
/usr/local/chronopolis/intake/dc/bridge-intake.jar /usr/local/chronopolis/intake/dc/bridge-
...
prepare
When running, the service will check for the following directories and create/apply permissions if they do not match:
- System Logging: /var/log/chronopolis/
- logging Journal data: /var/lib/chronopolis/data/
- journaled data for tokenization
Configuration
Depending on where the data is bound, there are several configuration options available
Chronopolis
Prefix
Specifying Replicating Nodes
DPN
Sample application.yml with all properties
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
# Cron timer for how often the bridge is polled bridge: poll: 0 0 0 * * * # Bagging Configuration bag: max-size: 2 unit: GIGABYTE dpn: node-address: University of California, San Diego, 9500 Gilman Dr, La Jolla, CA 92093 node-contact: Sibyl Schaefer node-email: - sschaefer@ucsd.edu - chronopolis-support-l@mailman.ucsd.edu # General chron configurationGeneral chron configuration ## chron.node: obsolete; ## chron.workDirectory: directory for storing information used for ongoing operations, e.g. file csv ## chron.stage.bags.posix.id: the id of the bag staging area for this service ## chron.stage.bags.posix.path: the path on disk to the bag staging area for this service chron: node: chron bagsworkDirectory: /exporttmp/bagschronopolis tokensstage.bags: /export/tokens preservationposix.id: /data/preservation4 restoration posix.path: /exportscratch0/restore prefix: bridge- ingest: endpoint: http://localhost:8081/ username: ingest-admin password: replace-me replicatingTo: - ucsd - umiacs # DPN Configuration - the server to use when creating replications # and connection information for the registry dpnReplicationServer: dpn-staging.ucsd.edu dpnbags # Chronopolis Ingest API Configuration ## ingest.api.endpoint: the http endpoint of the Ingest Server ## ingest.api.username: the username to connect to the Ingest Server with ## ingest.api.password: the password to connect to the Ingest Server with ingest.api: endpoint: http://localhost:8000/ username: admin password: admin # ACE IMS Configuration ## ace.ims.endpoint: the fqdn of the ims ## ace.ims.port: the port to connect to the ims on; one of: 80, 443 ## ace.ims: endpoint: http://localhost:3000/ usernameport: chron80 api-keywaitTime: replace-me 6000 # Duracloud Bridge Configuration # The storage areas the bridge app writes in to # Connection information to query the bridge ## duracloud.bridge.name: an identifier for the bridge ## duracloud.bridge.snapshots: the path on disk to the snapshots directory ## duracloud.bridge.restores: the path on disk to the restores directory ## duracloud.bridge.username: the username to use when querying the bridge ## duracloud.bridge.password: the password to use when querying the bridge ## duracloud.bridge.password: the http endpoint of the bridge duracloud: bridge: - name: bridge-0 snapshots: /exportbridge-0/snapshots/ restores: /exportbridge-0/restore/ bridge: username: bridgereplace-me password: replace-me endpoint: http://localhost:8080/ # Push settings to decide what networks to push the snapshots into pushDPN: false pushChronopolis: false - name: bridge-1 snapshots: /bridge-1/snapshots/ restores: /bridge-1/restore/ username: replace-me password: replace-me endpoint: http://localhost:8081/ # Logging configuration logging: file: /var/log/bridgeintake/intake.log level: org.springframework: ERROR org.hibernate: ERROR org.chronopolis: debug org.chronopolis.intake.duracloud.config: trace # Extraneous settings # automatic cleaning of staging areas (not well tested) # only perform dry-runs when the Cleaner runs # disable SNI on https connections - false is the default and recommended value cleanerEnableddisableSNI: false cleanDryRun: false disableSNI: false |
...
Release Notes
Release 3.0.1
09 April, 2019
- Bug Fix: Resolve error with startup failing to configure our beans
Release 3.0.0
21 March, 2019
- Remove deprecated DPN workflow
- Update .gitlab-ci.yml to cache maven artifacts
Release 2.4.3
21 March, 2019
- Update Chronopolis and chron-test dependencies to latest
Release 2.4.2
23 January, 2019
- Bump Chronopolis dependency version
- Bug Fix: Fix csv upload to create the MultiPart data correctly
Release 2.4.1
22 January, 2019
- Bump Chronopolis dependency version
Release 2.4.0
11 January, 2019
- Bridge consolidation work to bring multiple bridge querying to a single service
- Update notification of missing Depositors to fire only once for the lifetime of the bridge
- Fix Bug resulting in successive snapshot runs not being processed if a check against the Depositor fails
- Update ChronopolisIngest for 3.0.0 workflow
Release 2.3.2
09 August, 2018
- Fixed bug which was preventing deserialization of certain classes
- Temporarily disabled HttpFilter because of bug in Ingest Server
Release 2.3.1
07 August, 2018
- Updated logic for appending a prefix to a depositor so that it is used in every operation
Release 2.3.0
10 July, 2018
...