Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Danny Bernstein  
  2. Peter Winckles (star)
  3. Andrew Woods (out)
  4. David Wilcox (out)  
  5. Peter Eichman
  6. Joshua WestgardBen Pennell (out)
  7. Jared Whiklo
  8. Bethany Seeger Paul Cummins
  9. Youn Noh
  10. Thomas Bernhart (might have to leave early)
  11. Ben Cail
  12. Rosie Le Faive
  13. Daniel Lamb
  14. Aaron Birkland

Part 2: 

Agenda

  1. Announcements
  2. Sprint Planning
    1. 6.0 Architecture Review
    2. Coming to consensus on:
      1. Definition of "rebuild"
        create all necessary indexes, caches, or other ancillary data structures derived from persisted content  (OCFL + unversioned content)
    3. unversioned content  
      1. What use cases must we support?
        1. Automatic versioning of staged content after X minutes/hours/days/weeks?
        2. Permanently unversioned content
        3. ?
      2. Issues to gain consensus on: 
        1. Are we agreed that unversioned content will live outside OCFL Storage Root?
        2. ?
      /staged content
      1. what shall we call it:  staged or unversioned  
    4.  Multi-object transaction implementation ideasapproach? 
    5. Transaction Sidecar Spec Update
    6. Major Areas of Work
      1. Design/Development
        1. Interface Definition
          1. Persistence API
          2. ?
        2. OCFL Client Development
          1. OCFL Java API
          2. OCFL Java Client Implementation
        3. Transactions
      2. Documentation
      3. Testing
      4. Import/Export/Migration
    7. Sprint Goals
  3. Status on organizing a Fedora documentation review
  4. Introduction to running Fedora with Valkyrie  
  5. Applying a digital preservation framework (e.g. NDSA Levels of Digital Preservation) to Fedora 6 
  6. Update on Fedora 6 Pilots 
  7. Status
    1. API Test Suite PRs
      1. https://github.com/fcrepo/Fedora-API-Test-Suite/pulls
    2.  Minimal 4 →5 migration needs testing  and code review:
      1. https://github.com/fcrepo4-exts/fcrepo-upgrade-utils/pull/17
  8. Your topic here...

...

  1. In Review

    Expand

    Jira
    serverDuraSpace JIRA
    jqlQueryfilter=13100
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


  2. Please squash a bug!

    Expand

    Jira
    serverDuraSpace JIRA
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    maximumIssues20
    jqlQueryfilter=13122
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


  3. Tickets resolved this week:

    Expand

    Jira
    serverDuraSpace JIRA
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    maximumIssues20
    jqlQueryfilter=13111
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


  4. Tickets created this week:

    Expand

    Jira
    serverDuraSpace JIRA
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    maximumIssues20
    jqlQueryfilter=13029
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5


Notes

PART 1

OCFL Java Client  - let's get Peter Winckles and Aaron Birklandtogether for a discussion about a common java ocfl client interface / interaction model - hopefully we can discuss in part two of today's meeting.

Designed outcome:  a java interface that will support both the java client and a JNI-wrapped Go Client.

Bethany Seeger :  will take a look at Greg Jansen 's PR.

Danny Bernstein  to add a discussion point around API Test Suite modifications and overall spec compliance discussion.

Joshua Westgard  to look into fcrepo-upgrade-utils PR  the next week.

Danny Bernstein  to run upgrade routine on  sample (e.g. plant patents ) datasets

Paul Cummins  raising the concern about bloat with respect automated versions.  His Fedora 3 application performs frequent updates to binary content.   In a world of version on change,  he would quickly experience bloat.   "Copy into new OCFL object + Delete old OCFL object" mechanism for removing unwanted object history is an acceptable solution.

There was shared on the understanding of the new versioning proposal and its implications. 

PART 2

Ben Pennell , Peter Winckles  and Danny Bernstein  continued the discussion of the new versioning proposal.

Everyone seemed concerned that this would represent a significant departure from the current interaction model of Fedora.

Also storing "significant version" files in OCFL seems a bit convoluted.  In order to restore a previous version,  clients would need to indicate whether or not to make the restored version "significant".   Fedora or possibly the client would then need to remove or keep that marker file accordingly.

Question:  What is the standard for rebuildability from OCFL?   Are we still in agreement that you should be able to take an OCFL layout,  copy it to a new location, stand up a new fedora instance against it, and have the state of Fedora be exactly what it was before the move?  Can any non-derivative state live outside OCFL? 

...

Part I

  1. Announcements:
    1. Good, well attended meeting in Switzerland. Interest in Fedora 6.
  2. Value prop of using Valkyrie with/without Fedora
    1. Fedora 4 & 5 about standardization
      1. Some performance issues
    2. Fedora 6 adding more preservation minded features
      1. Transparent, human-readable filesystem
      2. Rebuildability from FS
    3. Why better to use Fedora over Preservica?
      1. Preservica not active storage
    4. Fedora offers standard middleware integrations: notifications, robust api
  3. Definition of Fedora rebuild
    1. Ability to recreate everything needed to run Fedora based on the files on disk (Daniel's exact words should be added here)
  4. Support unversioned content?
    1. Some use Fedora to author content over a period of time. They want to version when the content is ready and not have a version of every change.
      1. Clean version set
      2. Reduce bloat on disk
    2. Fedora currently offers the ability to mutate objects and create fixed versions. In OCFL, everything is immutable. How to reconcile this, and maintain Fedora functionality.
      1. Store everything in OCFL and make version logical structures on top of this
      2. Store only immutable content in OCFL
        1. Mutable content stored within OCFL's 'deposit' directory
        2. Fedora maintains mutable content
      3. Automatically periodically "flush" staged changes to OCFL
        1. OCFL versions would not be meaningful – requires logical versions
    3. There's a possibility that an object could never have a version. Is this okay?
    4. Auto-versioning should be toggle-able
    5. Islandora controls versioning – auto-versioning would be off by default
    6. Where does unversioned content live?
      1. OCFL 'deposit' directory is intended for short-lived version creation and not long-term storage
      2. OCFL 'deposit' directory space is out of spec and is up to the individual library to use it as it will
    7. Currently, you can delete version markers, but this is not possible if you store version markers in OCFL. Perhaps store version markers outside of OCFL?
      1. All changes stored in OCFL
      2. Fedora managed version file that maps OCFL versions to logical versions. This mapping is outside of the OCFL object.
    8. Pros about versioning every change in OCFL
      1. Easier to implement
      2. Easier to reason about
      3. Rebuildability is easier
    9. Cons about versioning every change in OCFL
      1. Binary bloat
      2. Inventory bloat
    10. OCFL versions are not presented in the API only Fedora versions are

Part II

  1. What is "auto-versioning"?
    1. On close of Fedora transaction an OCFL version is created – this happens regardless of whether or not auto-versioning is enabled
    2. Auto-versioning on: Fedora tag automatically created for every OCFL version
    3. Auto-versioning off: No Fedora tag created
    4. Absent a tag file that describes Fedora versions, OCFL versions are exposed
    5. Counter-prop: No auto-versioning. All Fedora versions must be manual.
    6. Cannot support tagging old OCFL versions with Fedora versions
    7. Concerns about exposing OCFL versions through Fedora APIs – don't want Fedora to be tied to OCFL
    8. How does this work when importing from old versions of Fedora?
      1. Fedora 4 & 5: can the versions be replayed?
      2. Fedora 4 → 5: Export 4, run transform, Import 5
      3. Fedora 5 → 6: Export 5, run transform that produces OCFL, point Fedora 6 to OCFL
      4. Fedora 3 → 6: Conversion of Fedora 3 files on disk to OCFL, point Fedora 6 to OCFL
  2. Transactions
    1. Transactions at the object level are do-able, but across multiple objects is trickier. May not need to implement multi object transactions.
    2. Bound to an archival group. Don't allow multi object transactions across different archival groups.
    3. Transaction endpoint exposed for every OCFL object
    4. Cannot make changes to objects out of scope of original transaction.
    5. How do you open a transaction for a new OCFL object?
      1. Create an empty object, open a transaction, update object, close transaction – would not remove the empty object on rollback
    6. Why limit transactions to just one object?
      1. Multi-object OCFL transactions are hard to implement
    7. Something in the import/export group for mapping to archival groups

Actions

  •  Aaron Birkland  to look explore notion of OCFL client with database as authoritative metadata source + asynchronous writing of the inventory.json file
  •  Peter Eichman   and maybe Ben Pennell to make recommendations re transaction side car specification.
  •  Andrew Woods will look into java 11 transition

...