Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The Bridge Intake Service provides data pulled down from the Duracloud Bridge App a path into Chronopolis. It currently handles bagging of snapshots, which includes validating files from the manifest given by Duracloud.

Links

Installation

Prereqs

  • Running Duracloud Bridge App to connect to
  • Staging area for creating bags

Install

  1. Get the latest rpm from http://adaptci01.umiacs.umd.edu/resource/bridge-intake/master/
  2. yum install

RPM

Installation can be done through the provided rpm. This offers builds for both RHEL6 and RHEL7.

Installation Notes

Installed Files For RHEL6 

/etc/init.d/bridge-intake
/usr/local/chronopolis/intake/dc
/usr/local/chronopolis/intake/dc/application.yml
/usr/local/chronopolis/intake/dc/bridge-intake.jar

Installed Files For RHEL7Installed files are as follows:

/usr/lib/systemd/

...

system/bridge-intake.

...

service
/usr/local/chronopolis/intake/dc
/usr/local/chronopolis/intake/dc/application.yml

...


/usr/local/chronopolis/intake/dc/bridge-intake.jar
/usr/local/chronopolis/intake/dc/bridge-

...

prepare

When running, the service will check for the following directories and create/apply permissions if they do not match:

  • System Logging: /var/log/chronopolis/
  • logging Journal data: /var/lib/chronopolis/data/
  • journaled data for tokenization

Configuration

Depending on where the data is bound, there are several configuration options available

Chronopolis

Prefix

Specifying Replicating Nodes

DPN

Sample application.yml with all properties

Code Block
languageyml
titleapplication.yml
collapsetrue
# Cron timer for how often the bridge is polled
bridge:
  poll: 0 0 0 * * *
 
# Bagging Configuration
bag:
  max-size: 2
  unit: GIGABYTE
  dpn:
    node-address: University of California, San Diego, 9500 Gilman Dr, La Jolla, CA 92093
    node-contact: Sibyl Schaefer
    node-email:
      - sschaefer@ucsd.edu
      - chronopolis-support-l@mailman.ucsd.edu

# General chron configurationGeneral chron configuration
## chron.node: obsolete;
## chron.workDirectory: directory for storing information used for ongoing operations, e.g. file csv
## chron.stage.bags.posix.id: the id of the bag staging area for this service
## chron.stage.bags.posix.path: the path on disk to the bag staging area for this service
chron:
  node: chron
  bagsworkDirectory: /exporttmp/bagschronopolis
  tokensstage.bags:
  /export/tokens
  preservationposix.id: /data/preservation4
  restoration  posix.path: /exportscratch0/restorebags
  prefix: bridge-
  ingest:
    endpoint: http://localhost:8081/
    username: ingest-admin
    password: replace-me
  replicatingTo:
    - ucsd
    - umiacs

# DPN Configuration - the server to use when creating replications
#                     and connection information for the registry
dpnReplicationServer: dpn-staging.ucsd.edu
dpn
# Chronopolis Ingest API Configuration
## ingest.api.endpoint: the http endpoint of the Ingest Server
## ingest.api.username: the username to connect to the Ingest Server with
## ingest.api.password: the password to connect to the Ingest Server with
ingest.api:
  endpoint: http://localhost:8000/
  username: admin
  password: admin
 
# ACE IMS Configuration
## ace.ims.endpoint: the fqdn of the ims
## ace.ims.port: the port to connect to the ims on; one of: 80, 443
##
ace.ims:
  endpoint: http://localhost:3000/
  usernameport: chron80
  api-keywaitTime: replace-me
6000
 
 
# Duracloud Bridge Configuration
# The storage areas the bridge app writes in to
# Connection information to query the bridge
## duracloud.bridge.name: an identifier for the bridge
## duracloud.bridge.snapshots: the path on disk to the snapshots directory
## duracloud.bridge.restores: the path on disk to the restores directory
## duracloud.bridge.username: the username to use when querying the bridge
## duracloud.bridge.password: the password to use when querying the bridge
## duracloud.bridge.password: the http endpoint of the bridge
duracloud:
  bridge:
    - name: bridge-0
      snapshots: /exportbridge-0/snapshots/
      restores: /exportbridge-0/restore/
  bridge: 
    username: bridgereplace-me
      password: replace-me
      endpoint: http://localhost:8080/

# Push settings to decide what networks to push the snapshots into
pushDPN: false
pushChronopolis: false    - name: bridge-1
      snapshots: /bridge-1/snapshots/
      restores: /bridge-1/restore/
      username: replace-me
      password: replace-me
      endpoint: http://localhost:8081/

# Logging configuration
logging:
  file: /var/log/bridgeintake/intake.log
  level:
    org.springframework: ERROR
    org.hibernate: ERROR
    org.chronopolis: debug
    org.chronopolis.intake.duracloud.config: trace
 
# Extraneous settings
#   automatic cleaning of staging areas (not well tested)
#   only perform dry-runs when the Cleaner runs
#   disable SNI on https connections - false is the default and recommended value
cleanerEnableddisableSNI: false
cleanDryRun: false
disableSNI: false

...

Release Notes

Release 3.0.1

09 April, 2019

  • Bug Fix: Resolve error with startup failing to configure our beans

Release 3.0.0

21 March, 2019

  • Remove deprecated DPN workflow
  • Update .gitlab-ci.yml to cache maven artifacts

Release 2.4.3

21 March, 2019

  • Update Chronopolis and chron-test dependencies to latest

Release 2.4.2

23 January, 2019

  • Bump Chronopolis dependency version
  • Bug Fix: Fix csv upload to create the MultiPart data correctly

Release 2.4.1

22 January, 2019

  • Bump Chronopolis dependency version

Release 2.4.0

11 January, 2019

  • Bridge consolidation work to bring multiple bridge querying to a single service
  • Update notification of missing Depositors to fire only once for the lifetime of the bridge
  • Fix Bug resulting in successive snapshot runs not being processed if a check against the Depositor fails
  • Update ChronopolisIngest for 3.0.0 workflow

Release 2.3.2

09 August, 2018

...