Attendees
- Bill
- Chris
- Dan
- Danny
- Jonathan
To Do for 0.2
Development
Task |
Assignee |
Status |
|---|---|---|
OCS: Update the title of the objects being created by OCS to not use DuraCloud URL |
Dan |
|
Create collection object (dfr:import) to be used as base collection in SiDora |
Dan |
|
OCS: Update the object creation process to add a relationship (in RELS-EXT) to the dfr:import collection object |
Dan |
|
Get CloudSync to perform preemptive authentication when making calls to Fedora |
Chris |
|
OCS: Ensure that messages coming from DuraCloud pertaining to content in administrative spaces are ignored |
Bill |
|
: Jira Cleanup
Jira Item |
Task |
Assignee |
Status |
|---|---|---|---|
DFR-103 |
Close, create another for 0.3 |
Danny Bill |
|
DFR-93 |
Push to 0.3 |
Dan |
|
DFR-89 |
Close, indicate that Apache Camel is the choice |
Dan |
|
DFR-88 |
Close (indicating that we were able to handle mime), create new JIRA to hook in the format determination tool |
Dan |
|
DFR-76 |
Push to 0.3 |
Dan |
|
n/a |
Create Jira Task: Create hierarchy of collection objects to mimic the full directory path captured in DuraCloud content ID |
Bill |
|
n/a |
Create Jira Task: Handle duplicate messages in OCS |
Bill |
|
n/a |
Create Jira Task: Handle file deletion |
Bill |
|
Discussion
Development
- Clean up the objects being sent into SiDora
- Glaring issue: Includes full URL of file as the object title (pretty Object name)
- SiDora is very dependent on knowing what the parent object is
- One approach: Make a single collection object, then add all object under that collection (such as "Imported Objects")
- Could have some code in OCS to create the once collection object
- OCS needs to create the collection object, needs to capture the PID, needs to create a relationship for each file object
- For 0.2, will expect that the collection object will exist in SiDora (created by hand for now)
- dfr:import as the PID of this collection object
- Each object brought in has a content ID which describes a heirarchy, we have the option to create the heirarchy (this is an 0.3)
- Two code changes necessary in OCS:
- Fix object title (Dan)
- Add relationship to dfr:import (Dan)
- Need to create the dfr:import object to add to SiDora (Dan)
- Refactor brittle code
- Should be completed as part of #1
- Fix sync authentication issue (Chris)
- Sending content from Fedora back to DuraCloud
- Fedora's authentication is not correctly challenging for certain URLs
- Work around is to have CloudSync to authenticate preemtively
- New versions of apache httpclient doesn't make this very easy
- Stop infinite object ingest loop
- Need to make sure we handle duplicate messages
- No guarantee that you won't get a JMS message twice
- Simple case, just make sure that items in the staging area are not duplicated
- Move to Jira
- Need to make sure that we skip messages that indicate objects added to DuraCloud admin spaces
- Can likely be handled with message selectors (Bill)
- Need to make sure we handle duplicate messages
Testing
- Test OCS in the cloud on Tomcat
- Set up and verify the full hookshot
- File on local file system, pushed to DuraCloud via DfR sync tool
- DuraCloud ingest message received by OCS and Fedora object created
- Fedora object stored in SiDora Fedora
- Fedora object discoverable via SiDora UI
- Fedora object synced back to DuraCloud via CloudSync
Current runtime setup
- DuraCloud instance: dfrtest.duracloud.org (two spaces, researcher-data and fedora-objects)
- upsync goes to researcher-data, CloudSync pushes to fedora-objects
- CloudSync running on an AWS instance (duraspace infrastructure AWS acct, using keys for DfR)
- OCS
- Plan to install OCS on the same server as CloudSync
- OCS currently running on Dan's machine
- To run locally: mvn jetty:run
- Need to have: ext.properties (not in github, due to passwords)
Release
- Tag codebase
git tag -a v0.2.0 -m "Tag for version 0.2.0" git push --tags origin master
- Update code for next version
mvn versions:set -DnewVersion=0.3.0-SNAPSHOT git push origin master
