Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Attendees - 

Regrets

Goals

Discussion items

TimeItemWhoNotes
5 minHousekeeping and updates

Next in-person meeting:

  • Sept. 24 - 26
  • Will be 2.5 days. Please do not schedule to leave before 12pm on the 26th if you are a Core Team member. The first half of the meeting will involve incorporating community feedback into specifications and working to finalize the drafts. The second half will be focused on the user interface design. 
  • Please contact Sibyl if you will miss any of the meeting time so catering can be accounted for.
  • Susan has created trips for everyone traveling. Please let me know if you have not received an email with a trip number from her. 
5 minUpcoming timeline

Aug 1: Specifications shared with Samvera and DDP communities

Aug and Sept Group meetings: Continue review of specifications and evaluation against user stories

Sept 24 - 26: In-person meeting

5 minCommunity involvement 
  • User stories responses - 11 views
  • https://docs.google.com/document/d/10vg-8Ago8cQsp4ZZyADSBl4El3oHMc9V1sfdemoISjY/edit?ts=5d360f9c 
    • Esme Cowles - Princeton
    • Nathan Tallman - Penn State
    • Nicholas Taylor - LOCKSS, Stanford
    • Handful of added user stories - would be beneficial if we had a user story meeting to review. Jessica will get a call on the calendar. Will do it after the next round of calls for feedback. 
  • Plan for drafted specifications
    • Comments due August 31, 2019.

      • Monday, August 5th, 2019 - Email to samvera-community, savera-partners, and samvera-tech. Post in Slack. Sibyl will send to DPscollab.

      • Wednesday, August 14th, 2019 - Follow-up email and post to Slack for feedback.

      • Thursday, August 22nd, 2019 - Final email and post to Slack for feedback.





15 min

Overview of proposed architecture 

Notes:

  • Specification Flow Diagrams - need to be updated. First link is much more current. Narrative is based on the API. 
  • Two main components of system that weren't necessarily named during the in-person - Gateway and Bridge. Gateway sits next to Repository, Bridge next to DDP.
  • Initialize - set up to allow communications to occur. Making calls to Bridge to add acount, Gateway would make additional calls to indicate where call back should go.
  • Specification Preservation Flow Select objects/work to transfer, using "Filegroup" as generic term.
    • Versioning question. Much of API between repo and gateway is subset/superset of S3. Look like version-enabled objects in S3. Partial update structure needs to be in place. Questions still surround updating smaller pieces - metadata file, for example.  
    • Deposit and Delete flows - a lot of similarity between. 
    • Assumptions around deletes being entire works/objects, should we be mapping that more closely in the call. Question - do we allow for deletion of entire works/objects and/or single files? Note - not a great deal of extra work needed to allow for both at DDP end. APTrust allows for deletion of single file, feature is used often. Problem with documentation related to deletion. Have to restore entire audit file to detail what happens. In this situation, that info would be stored in the repository (question) Repository side workflow and policy is not entirely thought out, but also the most straightforward. The kinds of events that trigger a new preservation version are relatively clearly defined. Process for deleting a file - delete from repo, trigger preservation event. If we are tackling partial object updates for versioning, then that process will be fairly straight-forward. ** Update user stories to include this. (Rosy will do).
    • Goal on Bridge side to be as generic as possible. Has to interact with Gateway and DDP. Calls are expected to be able to understood by a variety of systems. Goal to make clear, consistent. 
  • ? Always reference files as part of a work? Deposits are part of a filegroup. Do we want works/filegroups as a namespace? Reference file independently of their work?
    • Audit - synchronous call, different than the other, asynchronous calls
  • Gateway spec - essentially functions as a cache. Gives the repo a stable place to interact with objects. Makes some guarantees about ensuring objects make it to the DDP (or reports appropriate errors).  
    • Reconstruction - person has a list of IDs to reconstruct
    • Questions on large file deposit. Holy bags or bag profiles may be useful. 
    • Audit metadata - real time request. Analysis needed concerning what data. Assumption that it is data available in real-time or maintained in a cache. 
  • Not a notion of "bag" in the Bridge - should there be? 

Next meeting
  • Reviewing user stories against specifications. Anyone have a good way of doing this beyond reading the user story and going over how it would be satisfied in the spec?

...