Design

1. Driving principles:
   1. Minimizing change to the user via the API
   2. OCFL-isms not bleeding into Fedora API
   3. Rebuildability
   4. Compliance with OCFL
   5. Retain URLs of migrated Fedora resources
   6. Performance
   7. Reducing complexity of implementation

OCFL persistence

Architecture

1. Retaining HTTP layer of existing Fedora codebase
2. Replacing ModeShape persistence with OCFL storage
3. Support for three interaction models: 
   1. atomistic (implicit) - every LDP resource maps to an individual OCFL Object
   2. archive group - hierarchy of LDP resources map into a compound OCFL Object
   3. archival-part (implicit) - an LDP resource that is a constituent part of a compound OCFL Object
4. Eliminate "single-subject-restriction", i.e. support arbitrary RDF
5. Fedora-specific information to be stored in the OCFL Object in a ".fcrepo/" directory
   1. i.e. Which file is the description of another file
   2. i.e. While file is an ACL
6. Optimizing reads/lookups with an internal database
   1. proposed database model: https://docs.google.com/document/d/1MsMfhae3thmNdoFtnTUnII3mr_-OkllRs9PvgnY1fDY/edit

OCFL persistence

1. Support for both OCFL objects:
   1. created by Fedora
   2. pre-existing, created by another application

Pre-existing OCFL storage hierarchy

1. Fedora-imposed constraints
   1. The OCFL storage hierarchy must have a single, consistent "ocfl_layout" (i.e. the storage path mapping algorithm must be determinant)

Mapping between LDP and OCFL

Opt-in model

1. Fedora resources may be created with an optional "archive" interaction model provided via headers.  

2. New resources created via POST or PUT to the archive will be LDP contained by the archive and will be stored within the OCFL object representing that archive.

3. If a resource is created without the "archive" model, new resources created via POST or PUT will be LDP contained by the parent resource, but will be stored as separate OCFL objects.

Notes/Implications

1. At creation time, user establishes interaction model. Changing the model would require additional migration tooling.

Fedora-specific details

1. /content/.fcrepo directory
2. Hashing (SHA256) on LDP path of resource

Scaling

• stateless Fedora instance(s) will scale horizontally
• database can be clustered and/or moved to cloud  database service(RDS, Aurora, etc)
• start with file system, scale out to  cloud object store (s3)

Bulk ingest

To flavors of pointing a Fedora instance at an OCFL backend.

1. via OCFL
2. via Fedora OCFL 
   1. ie "content/.fcrepo" aware

https://docs.google.com/document/d/15QwmyNFMopJW4MHxQNTiOGh_WXsOPr9-yzctGLCgT5k/edit

Performance

1. Many members: performance should improve significantly since list of members will be supplied by a database index (which should support a degree of in-memory caching).  No loading of modeshape nodes required.

Open questions

1. Role of OCFL storage roots
   1. Could be valuable for multi-tenancy, but client interaction model has not been detailed
2. What is the mapping / algorithm / relationship between:
   1. Fedora URL of LDP resource
   2. OCFL Object.ID
   3. OCFL storage path for associated OCFL Object

Implementation notes

1. Provide new implementation of fcrepo-kernel-api that interacts with OCFL persistence
2. Interactions with OCFL persistence should initially take advantage of the JHU OCFL client

Prototyping proposal

1. Expose JHU OCFL client functionality with minimal HTTP endpoints
   1. Such an endpoint should implement minimal LDP interactions
2. Use HTTP over OCFL to test:
   1. Performance bottlenecks
   2. Scale viability (e.g. NLM migration)
   3. User expectations, ergonomics

Versioning

1. Support for both versioning models:
   1. version an object on-demand (manual versioning)
   2. version an object on-change (auto-versioning)
2. Support for toggling auto-versioning on/off
3. One-to-one correspondence between OCFL versions and mementos
4. For archival-groups, any new version of the OCFL Object captures current state of the entire archival-group

Versioning on-demand

1. Same as Fedora 4 and 5 version creation: POST to a resource's "/fcr:versions" endpoint to create a Momento (i.e. a new OCFL version directory)
2. Actively edited objects not captured in a "cache/" directory at the sibling-level with OCFL version directories

Versioning on-change

1. Every update to a Fedora resource results in a new OCFL version directory
2. Potential downsides:
   1. Potential storage impact
   2. Potentially creates "noisy" version history
   3. Note: Transactions could mitigate "noisy" version history by grouping multiple updates in a single commit

Implementation notes

1. Same code logic used for creation of OCFL versions / Mementos in both on-demand and on-change models

Import / Export   (Migration)

• https://docs.google.com/document/d/1nvUDar1DbexdU3oTRGxzcZGG3qxsD2HWQbGyyRJIgDE/edit#

Fixity service

• Scheduled fixity service:
  • Probably not part of the core,  run as a separate service possibly controlled via camel.
• https://docs.google.com/document/d/1bTM0MEoaa88IFaT1zR6YlpFAnLOshjEIKi7miFMm620/edit#heading=h.9zyqk9cxc88

Query endpoint

Proposal: Query endpoint should support the following use cases:

• List all resources

• List resources by mimetype

• List resources by parent

• List resources by mimetype, parent, and modified date (<>=)

• List resources where modified  <> x date.

https://docs.google.com/document/d/13C_UhB37docTd8ZTD1ZSKuFhX6q9_3Boo67hjUC2YyM/edit

Transaction service

• No definitive decisions made.
• Proposal: no change to the Fedora API spec in 6.  We will either:
  • align code with the (as-yet-to-be-ratified) side-car specification
  • leave HTTP API unchanged while introducing the possibility of auto-versioning on transaction completion.

Raw notes

1. VA Beach Meeting notes